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Executive Summary 

The Study of School Turnaround (SST) examines the change process in a diverse, purposive sample of 
schools receiving federal School Improvement Grants (SIG) from 2010-11 to 2012-13. With the passage 
of the American Recovery and Reinvestment Act of 2009 (ARRA), the SIG program underwent three 
major shifts. First, ARRA boosted total SIG funding in fiscal year 2009 to approximately 6.5 times the 
original 2009 appropriation through Title I, section 1003(g) of the Elementary and Secondary Education 
Act (ESEA). SIG funds were distributed to states by formula based on each state's Title I share. States 
then had to competitively make SIG awards to districts with eligible schools. Second, ARRA targeted 
funds at only the very worst schools— those that were in the bottom 5 percent of performance and had 
been low performing for an extended period of time. Third, schools receiving SIG were now required to 
implement one of four prescriptive intervention models believed to be more aggressive and 
comprehensive than those generally adopted under prior policies (Flurlburt, Therriault, & Le Floch, 2012) 
(see Box ES.l). By increasing the level of funding, better targeting these funds to the persistently lowest- 
achieving schools, and requiring that schools adopt specific intervention models, the revamped SIG 
program aimed to catalyze more aggressive efforts to turn around student performance. This report 
focuses on a small sample of schools receiving SIG over the first three years of the revamped SIG 
program, from 2010-11 to 2012-13. It presents findings from the study's 25 core sample schools, which 
were the focus of data collection in spring 2011 and spring 2012, and a subsample of 12 of the 25 
schools (collectively referred to as the core subsample), which were selected for data collection in spring 
2013 and are the focus of more in-depth analyses looking across all three years of SIG. 

Overview of Final Report Findings 

• A majority of the 25 core sample schools replaced their principal (21 schools) at least once in 
the year before SIG (2009-10) or in Year 1 of SIG (2010-11). Two of the four SIG intervention 
models (transformation and turnaround) required the principal to be replaced either in the first 
year of SIG or in the year prior to SIG. By Year 2 of SIG (2011-12), 9 of the 25 core sample 
schools had replaced their principal twice; only three schools maintained the same principal 
across this three-year period. Of the 20 principals who were new to their schools in 2010-11 or 
2011-12, half (10) were described by respondents 1 as an improvement over their predecessors. 

• About half of the 25 core sample schools (12 schools, including 9 turnaround, 2 restart, and 1 
transformation) replaced at least 50 percent of their teachers during the 2009-10, 2010-11, or 
2011-12 school years. Respondents in 7 of these schools characterized the change in teachers 
as positive for the school, bringing new energy and improved morale. During the first two years 
of SIG, almost all core sample schools (24) created new non-teaching positions. The most 
commonly reported new non-teaching positions were instructional, technology, and data 
coaches (14 schools), followed by additional school administrators (11 schools). In the second 
year of SIG, the principal and district officials in about three fourths of the core sample schools 
(18 schools) indicated that recruitment and/or retention challenges limited the school's ability 
to build a skilled and motivated staff. 


1 This analysis, and others in which respondents report on changes from the prior year, excludes newly-hired 
school staff. 
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• According to teacher survey data, more teachers reported participating in professional 
learning on math, literacy, and data use than on ELL instruction, special education, or 
classroom management during Year 2 of SIG (2011-12). On average, surveyed teachers in two 
thirds of the core sample schools reported spending a larger proportion of their hours in job- 
embedded professional learning activities than in more traditional activities that year. 2 In 17 of 
21 core sample schools with sufficient data, most teacher survey respondents reported learning 
and changing their practice after participating in professional learning on math, literacy, or data 
use. 

• Core sample schools reported receiving support from their district (22 of 22 schools 3 ) and 
external support provider(s) (22 of 25 schools), but in some cases, respondents described 
shortcomings in their district or external support. Respondents in 10 of 22 core sample schools 
with sufficient data perceived their district's support efforts to be useful to their school's 
improvement efforts. 

• Among the 12 core subsample schools, those that appeared to engage in more efforts to build 
human capital in Years 1 and 2 of SIG (7 schools) were more likely to improve their 
organizational capacity (or sustain their already higher capacity). Additionally, most teachers in 
7 of the 12 core subsample schools reported on the Year 3 teacher survey that their school had 
changed in primarily positive ways during the three years of SIG. The quality of leadership in the 
12 core subsample schools appears to be related to teachers' perceptions of school 
improvement, based on data from the teacher survey and site visits. 

• Sustainability of any improvements may prove fragile. Core subsample schools that had higher 
levels of organizational capacity by Year 3 of SIG (2012-13) also had higher scores on the 
teacher survey scale measuring perceived sustainability. Of the 12 core subsample schools, 
which we followed for all three years of SIG, 2 schools appeared to have strong prospects for 
sustainability, 6 schools appeared to have mixed prospects for sustainability, and the remaining 
4 schools appeared to have weak prospects for sustainability, according to teacher survey 
responses and site visit data. 

These findings build upon an analysis of the change process in the case study schools following the first 
year of SIG implementation (2010-11) (Le Floch et al., 2014). Analysts used the data collected over the 
subsequent two years of SIG (2011-12 and 2012-13) to understand the ways in which the trajectory of 
these schools shifted over time. For most of the remainder of the Executive Summary, we discuss these 
key findings in additional detail by reviewing each main findings chapter in the report (Chapters 3-9). 
Chapters 3-7 present analyses of school efforts to build human capital among the 25 core sample 
schools. Chapters 8-9 take a closer look at the change process in 12 of the 25 core sample schools (core 
subsample) over all three years of SIG. 


2 The analyses of teachers' professional learning opportunities were designed to provide a snapshot of the 
professional learning activities that took place in core sample schools during Year 2 of SIG (2011-12). The study did 
not collect data on teachers' professional learning activities prior to SIG to address whether the professional 
learning activities that teachers reported in Year 2 were associated with SIG-supported school turnaround efforts. 

3 The three restart schools in the core sample were excluded from analyses of district support because they are 
managed by EMOs or CMOs. 
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Study Purpose and Methodology 

SST is a set of case studies that documents the change process during a three-year period in SIG schools 
located in diverse state and local contexts. The case studies are designed to describe the characteristics 
of the schools, the decisions and strategies that the schools and their districts undertake, and the 
challenges they face in attempting to dramatically improve school performance. The study is based 
primarily on interviews and surveys, which, by nature, are self-reported. These data were not audited or 
validated with external sources. SST does not examine the impact of SIG on student achievement 
outcomes because the case study approach is not well-suited to investigate these questions rigorously, 
and, because of the limited sample size, it is not designed to document the practices of all, or even 
necessarily a representative sample of, SIG schools nationwide. 4 Rather, SST is an in-depth examination 
of how SIG funds and strategies have evolved in a small but diverse group of SIG schools over the three 
years of funding (2010-11 through 2012-13). 


Box ES.l. Detail on SIG Program 

According to the final rules issued by the U.S. Department of Education (ED) for the SIG program, 
persistently lowest-achieving schools are eligible to receive SIG and include a state's lowest- 
performing five percent of schools or five schools, whichever number is greater, in terms of overall 
academic performance for all students, and schools that exhibit a lack of progress toward 
achievement goals. SIG defines three eligibility tiers for persistently lowest-achieving schools, with 
Tier I and Tier II representing the highest priority for SIG funding, and Tier III representing the lowest 
priority. One of four intervention models must be specified for implementation in each Tier I and 
Tier II school identified in a district's SIG application to its state for funding (Hurlburt, Therriault, & 

Le Floch, 2012). The key requirements for each model are as follows: 

Turnaround model. Replace the principal and no less than 50 percent of the staff, introduce 
significant instructional reforms, increase learning time, and provide the school sufficient 
operational flexibility (e.g., staffing, time, and budgeting) and support (e.g., ongoing, intensive 
technical assistance and related support). 

Restart model. Reopen the school under the management of a charter school operator, a charter 
management organization (CMO), or an education management organization (must enroll, within 
the grades served, any former student who wants to attend the school). 

School closure. Close the school and reassign students to higher-achieving schools. 

Transformation model. Replace the principal, develop a teacher and leader evaluation system that 
takes into account student progress, introduce significant instructional reforms, increase learning 
time, and provide the school sufficient operational flexibility and support. 

These models are consistent with those defined in other ARRA-funded initiatives, including Race to 
the Top and the State Fiscal Stabilization Funds, Phase 2. For more information on SIG 
requirements, see ED's webpage on SIG legislation, regulations, and guidance 
(http://www2.ed.gov/programs/sif/legislation.html). 


4 IES is conducting an evaluation of SIG (Implementation and Impact Evaluation of Race to the Top and School 
Improvement Grants) to examine the impact of SIG on school and student outcomes using a larger sample of 
schools and methods more appropriate for estimating impacts and how they are correlated with various broader 
measures of implementation. These findings are anticipated by the end of 2016. 
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The study team collected data from stakeholders at the state, district, and school levels. The school 
sample was selected to include variation in state, district, and school characteristics hypothesized to be 
associated with implementation patterns and turnaround success. Analysts initially identified a base 
sample of 60 schools from the cohort of schools awarded SIG funds in summer 2010. Closure schools 
were not included, and restart schools were oversampled. The final base sample includes turnaround, 
restart, and transformation schools, with the majority of the sample being transformation schools (as it 
is in SIG-funded schools nationwide). From this base sample of 60 schools, we selected three 
subsamples: the core sample, the rural sample (see Rosenberg et al., 2014), and the sample of schools 
with a high proportion of English language learners (ELLs) (see Boyle et al., 2014; Golden et al., 2014). 
This report is based on the findings from the 25 core sample schools, which were the focus of data 
collection in spring 2011 and spring 2012, and from a subsample of 12 of these 25 schools (core 
subsample ), which were selected for data collection in spring 2013 and as the focus of more in-depth 
analyses looking across all three years of SIG. 

The data collection in spring 2011, 2012, and 2013 included a teacher survey and a site visit from two 
study team members to conduct interviews and focus groups with a range of district and school 
stakeholders, including district officials (i.e., superintendents, SIG directors, and other district 
personnel), principals, teachers, instructional coaches, school improvement teams, external support 
providers (i.e., curriculum/instructional providers, school turnaround organizations, CMOs), union 
representatives, students (in high schools only), parents, and community members. The complete set of 
data collection instruments can be found at http://www.air.org/project/study-school-turnaround. 

The site visit data were analyzed by coding transcribed interview notes using Atlas.ti® (a qualitative 
software program) and compiling the results into an online data repository. The teacher survey data 
were then used in conjunction with the qualitative data to examine patterns by school level, SIG 
intervention model, and other school characteristics (see Chapter 2 for a more detailed discussion of the 
sample selection, data collection activities, and analytic procedures). 

Throughout this report, we incorporate direct quotations from study respondents. There are two 
primary reasons for the inclusion of quotations: one methodological and the other stylistic. With regard 
to methodology, by providing example quotations with explanations of our analytic measures rather 
than merely describing these measures in the abstract, we can more concretely illustrate how analysts 
coded the raw data. This approach lends more transparency to how the measures were constructed and 
allows the reader to better judge whether the measures appear well grounded. With regard to style, 
direct quotations enhance the clarity and relevance of the study, which is based largely on qualitative 
data. These data provide uniquely detailed, contextual information that can convey meaning through 
illustrative examples. Quotations were purposefully selected to enrich the findings that were arrived at 
through systematic, carefully documented analyses. These quotations are not representative of all of 
our data and are only meant to enrich a particular finding, not formally justify it. 

Overview of Year 1 Findings 

Data collection began during the first year of funding for Cohort I SIG schools (2010-11) in an effort to 
understand the characteristics of the case study schools, the decisions and strategies that the schools 
and school districts undertook (and why), and the challenges faced by stakeholders as they attempted to 
dramatically improve school performance. The Year 1 report (Le Floch et al., 2014) addressed questions 
related to the context of the case study schools, the role of school leaders in Year 1 of SIG, initial 
improvement actions undertaken by SIG schools after being awarded the grant, the role of SIG in the 
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initial change process, organizational capacity at the end of Year 1, and perceived improvement during 
the first year of implementation. 

Five key findings emerged from the analysis of activities in these 25 schools during the first year of SIG: 

• Although all were low-performing, core sample schools differed in their community and fiscal 
contexts, their performance and reform histories, and their interpretations of the causes of— 
and potential solutions for— their performance problems. 

• Approaches to leadership varied across the set of core sample schools with most principals 
exhibiting a mix of leadership qualities. The most frequently reported leadership approach 
among the core sample schools was transformational leadership, referring to principals who can 
develop leaders and motivate and engage their staff behind a strong organizational vision. 
Although respondents in the majority of schools reported some improvement in 2010-11, 
schools in which respondents described the improvements in the greatest number of areas 
also had higher levels of principal strategic leadership (referring to principals who are able to 
formulate a strategy for school improvement and translate that strategy into concrete priorities 
and specific actions) and were more likely to have experienced a disruption from past 
practices. 

• For most of the core sample schools, respondents did not perceive SIG as the primary impetus 
for the change strategies that had been adopted. In 19 of these schools, the improvement 
strategies and actions implemented during the first year of the grant (2010-11) were reportedly 
a continuation of activities or plans that predated SIG. 

• At the time of data collection, 7 of the 25 core sample schools had experienced a visible 
disruption from past practice. The remaining schools appeared to be following a more 
incremental approach to improvement. 

• Overall, core sample schools with the lowest levels of organizational capacity in 2010-11 were 
those in which teachers reported having fewer resources, the SIG award represented a larger 
percentage of the prior year's per-pupil expenditure, and respondents perceived the SIG 
award as a catalyst for change. 

Key Final Report Findings 

Building Human Capital through Leadership Strategies (Chapter 3) 

Case study research posits that principals hold a central role in leading major school reform efforts 
(Edmonds, 1979; Hassel & Flassel, 2007; Flerman et al., 2008; Johnson & Asera, 1999; Picucci et al., 2002; 
Rhim et al., 2007; Whiteside, 2006). The SIG program requires principal replacement as part of the 
turnaround and transformation models in schools where the principal has been in place for more than 
two years prior to SIG (U.S. Department of Education, 2010). 5 Empirical studies and theoretical 
explorations of leadership have asserted that leadership in a school setting may come from multiple 
sources and be distributed across multiple individuals and structures (Elmore, 2000; Spillane, Flalverson, 
& Diamond, 2004). We therefore examined the extent of principal replacement in the 25 core sample 
schools through the first two years of SIG implementation, as well as the ways in which principals in 


5 In addition, schools adopting the restart model often replace principals as they transition to a new governance 
structure. 
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these schools used distributed leadership to leverage the knowledge and skills of multiple staff 
members to enhance collective leadership capacity. 

A majority of the 25 core sample schools (21 schools) replaced their principal at least once in the year 
before SIG (2009-10) or in Year 1 of SIG (2010-11). Two of the four SIG intervention models 
(transformation and turnaround) required the principal be replaced either in the first year of SIG or in 
the year prior to SIG. By Year 2 of SIG (2011-12), 9 of 25 core sample schools replaced their principal 
twice; only three schools maintained the same principal over this three-year period. 

Of the 20 principals who were new to their schools in 2010-11 or 2011-12, half (10) were described as 
an improvement over their predecessors by teachers, instructional coaches, school leadership teams, 
or parents. Fourteen of the 20 new principals had between 5 and 10 years of experience as principals at 
others schools, in addition to longer-term experience as teachers and/or assistant principals in low- 
performing schools. 

School leaders' approaches to sharing leadership responsibilities among school stakeholders and 
engaging school staff in decision-making processes differed across the 25 core sample schools with 
respondents at 7 schools providing evidence of distributed leadership during Year 2 of SIG (2011-12). 

Respondents at 12 schools provided evidence of moderately distributed leadership, while respondents 
at the remaining 6 schools provided little evidence of distributed leadership. 

Respondents in over half of the 25 core sample schools (15 schools) reported that administrators 
made efforts to enhance distributed leadership during the first two years of SIG. Though these efforts 
differed by school, principals often promoted distributed leadership through the creation of new teams 
or committees, the addition of new staff to existing teams, leadership opportunities for teachers and 
coaches, and additional ways for school staff to provide input into the decision-making process. In the 
remaining 10 schools, the principals did not report developing distributed leadership during the first two 
years of SIG. 

Building Human Capital through Staffing Decisions (Chapter 4) 

SIG required that schools implementing the turnaround model replace at least 50 percent of 
instructional staff. This requirement rests on the assumption that the existing staff in persistently low- 
performing schools may not have the necessary knowledge, skills, or motivation needed to dramatically 
alter school performance, and that replacing them with more qualified individuals will help yield school 
improvement (Perlman & Redding, 2011). To gain a better understanding of staff replacement and its 
implications for building human capital, we examined school staffing strategies aimed at teachers and 
other school-level support staff in the 25 core sample schools through the first two years of SIG 
implementation. In addition, we explored the challenges of recruiting and retaining high-quality 
teachers among the core sample schools, and identified the supports that district provided to address 
those challenges. 

About half of the 25 core sample schools (12 schools) replaced at least 50 percent of their teachers 
during the 2009-10, 2010-11, or 2011-12 school years. For nine of these schools, the changes were 
made to comply with the requirements of the turnaround model being implemented; two schools did so 
as part of their new charter management organizations, and one school did so as part of the principal's 
school improvement plan. Respondents at seven schools characterized the change in teachers as 
positive for the school, bringing new energy and improved morale. 

During the first two years of SIG, almost all 25 core sample schools (24 schools) created new non- 
teaching positions. The most common new position were coaches (including instructional, technology, 
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and data coaches) (14 schools), followed by additional school administrators (11 schools). Schools also 
added other nonteaching positions, such as parent or community liaisons to coordinate activities and 
build relationships with parents and community-based organizations, social workers to address specific 
student needs such as homelessness, and technology coordinators. 

In about half of the 25 core sample schools (12 schools), principals provided evidence that staffing 
decisions were made with specific school needs or goals in mind. Purposeful staffing decisions included 
the implementation of strategies to minimize the impact of less effective staff and using student data to 
identify areas where more effective instruction is needed. Of the principals that did not provide 
evidence of purposeful approaches to staffing (13 schools), some described district policies that limited 
their ability to make intentional staffing decisions; however, these principals did not describe ways to 
maximize existing staff capacity to support students, or efforts to remove undesirable hires. 

In the second year of SIG, principals and district officials in about three fourths of the 25 core sample 
schools (18 schools) reported that recruitment and/or retention challenges limited the school's ability 
to build a skilled and motivated staff. Within these schools, principals and district officials most often 
(12 schools) attributed district-level conditions and policies to recruitment and retention challenges 
(such as layoffs, involuntary transfers, and hiring processes that limited teacher applicant pools). School- 
level challenges, such as stressful school environment, poor school reputation, or long commutes to 
school were reported less often (7 schools). Principal and district officials in the remaining seven core 
sample schools did not indicate that recruitment or retention presented a challenge for the school in 
Year 2 of SIG (2011-12); these schools all had stable teaching staffs during that school year. 

Principals and district officials in nine schools across seven districts reported that their districts 
provided advantages to SIG schools in the hiring process to better enable them to recruit qualified 
staff. In addition, teachers in eight schools across four districts reportedly received a monetary bonus 
for working at the school, 6 and principals in two schools reported extra supports for teachers to alleviate 
the expense of their long commutes to the school. 

Efforts to Build Human Capital Using Teacher Professional Learning Strategies 
(Chapter 5) 

In the first year of SIG (2010-11), the most common improvement action undertaken by the 25 core 
sample schools was to increase professional learning opportunities, including during the school day, 
during summer months, or on specific professional learning days (Le Floch et al ., 2014). This reform 
strategy is cited as an approach to improve student outcomes in low-performing schools (Opfer & 
Pedder, 2011) and is required for schools adopting the SIG turnaround and transformation models. We 
examined several aspects of professional learning in the 25 core sample schools through the first two 
years of SIG implementation that are intended to increase teacher learning and, ultimately, the human 
capital of the school. We explored the number of hours that teachers reported participating in such 
professional learning activities, the degree to which the activities appeared to be purposeful or tied to 
schools' goals, the extent to which schools balanced job-embedded professional learning activities with 
traditional ones, and, finally, how teachers reported change in their own practice based on their 
professional learning experiences. 


6 Other types of monetary benefits, such as performance-based incentives paid if certain student improvement 
goals are met and extra pay for working during extended hours or participating in additional professional learning 
opportunities, were not included in this analysis. 
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Teachers' participation in professional learning activities varied within and across schools based on 
Year 2 (2011-12) teacher survey data. Among the 25 core sample schools, the median number of hours 
ranged from 0 to 187 hours, with the difference between the 25 th and 75 th percentiles ranging from 36 
to 158 hours. 

More teachers in core sample schools reported participating in professional learning on math, literacy, 
and data use than on ELL instruction, special education, or classroom management. In both high 
schools and elementary schools, the most common professional learning topics were math and literacy 
instructional strategies and data use (see Exhibit ES.l). 


Exhibit ES.l. 

Percentage of Teachers Reporting Participating in Professional Learning, 
by Topic, 2011-12 


Topic 

Percentage of Elementary School 
Teachers Reporting Participating in 
Professional Learning 

Percentage of High School Teachers 
Reporting Participating in 
Professional Learning 

Teaching students with disabilities 

31.4% 

43.8% 

Teaching English language learners 

41.3% 

44.2% 

Classroom management 

44.2% 

45.2% 

Mathematics instructional strategies 

61.8% 

78.2% 

Literacy instructional strategies 

77.0% 

84.3% 

Data use 

80.2% 

62.2% 


Source: SST teacher survey, spring 2012. 

Notes: Includes 23 of 25 core sample schools (N = 794 teachers). At the high school level, the percentage of teachers reporting 
participation in "mathematics instructional strategies" includes only self-identified mathematics teachers, and the percentage 
of teachers reporting participation in "literacy instructional strategies" includes only self-identified English language arts 
teachers. Two schools were excluded from this analysis because they did not meet the 50 percent response rate threshold on 
the teacher survey. Professional learning hours categorized as "other" were excluded from this analysis. 


Principals and teachers in six core sample schools articulated connections between offered 
professional learning activities and the school's goals or needs. However, in the majority of schools (15 
schools), respondents offered mixed perceptions about whether their professional learning addressed 
the needs of the school. Respondents in four schools were not able to articulate connections between 
the activities offered and the school's needs. The activities appeared to be unrelated to the school's 
performance goals, improvement plans, or student needs. 

On average, surveyed teachers in nearly two thirds of the 25 core sample schools (15 schools) 
reported spending a larger proportion of their hours in job-embedded professional learning activities 
than in more traditional activities. Job-embedded professional learning activities included, for example, 
classroom coaching, structured common planning time, meetings with mentors, consultation with 
outside experts, and observations of classroom practice. On the other hand, traditional formats included 
teacher workshops, conferences, and college-degree courses, among others. These activities tended to 
be conducted outside the teachers' regular classroom setting and in concentrated blocks of time (e.g., 
over the summer or on a monthly basis). 

Respondents in the majority of the 25 core sample schools reported making efforts to enhance 
teacher learning by establishing policies, structures, and systems that would support data use (19 
schools) and teacher collaboration (16 schools) during the first two years of SIG. Across schools, 
approaches to support data use included establishing student data systems and incorporating 
benchmark assessments into instruction, among others. The most prevalent way that schools 
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established conditions to support teacher collaboration was to establish a protected time for teachers to 
work together across grades, within a grade, or within a subject. 

In 17 of 21 core sample schools with sufficient data, most teachers reported learning and changing 
their practice after participating in professional learning on math, ELA, or data use (the three most 
commonly reported professional learning topic areas). In 12 schools, teachers most commonly 
reported changing their practice in ELA. 

District and External Support Providers' Efforts to Build Human Capital in SIG 
Schools (Chapter 6) 

In addition to building the human capital capacity of SIG schools by changing the composition of staff 
(through replacement, recruitment, and retention strategies) and through professional learning, SIG 
guidance recognizes the important role that districts and external support providers play in the 
improvement efforts of SIG schools. Research on school improvement points to the important role of 
districts and external support providers in the reform process, explaining that external providers may 
provide knowledge, training, and services that school-level staff cannot (Finnigan, Bitter, & O'Day, 2009; 
Honig, 2004; Massell & Goertz, 2002; O'Day & Bitter, 2003; Supovitz, 2006; Zavadsky, 2012). To 
understand the ways in which districts and external support providers enhanced the capacity of the 25 
core sample schools - or the ways in which they did not - we explored the types of organizations that 
are partnering with SIG schools, the types of support being provided to them, the fit and intensity of 
such support (external support providers only), and the extent to which school-level respondents 
perceived the partnerships to be beneficial for their school's improvement efforts. 

Respondents from all 22 core sample schools included in the district support analyses 7 reported that 
their district provided them with at least one of the following: teacher professional learning activities 
(20 schools), principal professional learning activities (15 schools), supportive teacher staffing policies 
(14 schools), and structures and systems to support data use (13 schools). Examples of professional 
learning services provided by districts to SIG schools included workshops focused on specific district 
priorities such as data use or literacy/mathematics instructional strategies, as well as district-funded 
instructional coaching. 

Respondents in 9 of the 13 core sample districts, which served a total of 16 core sample schools, 
reported having sub-districts or designated staff positions in place to support low-performing schools 
during Year 2 of SIG (2011-12). Of these nine districts, four districts, serving nine core sample schools, 
established sub-districts, often referred to as "zones," whose role was oversight and support for the 
turnaround efforts in specific sets of low-performing schools, including SIG schools. Five districts, serving 
seven core sample schools, did not create new sub-districts but did establish central office positions or 
teams charged with overseeing and supporting SIG schools. Schools in these nine districts tended to 
report receiving support in more areas than schools in the remaining four districts, which reportedly did 
not have any specialized structures in place to support SIG schools. 

Respondents in 10 of the 22 core sample schools included in the district support analyses perceived 
their district's support efforts as useful to their schools' overall improvement efforts. In these schools, 
respondents spoke positively about their interactions with district staff, the accessibility of district staff, 
and the overall level and types of support they received from the district. Respondents in eight schools 


7 The three restart schools in the core sample were excluded from these analyses because they are managed by 
EMOs or CMOS. 
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expressed mixed perceptions of the district's efforts to support their school's improvement, while 
respondents in three schools indicated that district efforts (or lack thereof) were constraining school- 
level improvement efforts. The one remaining school was excluded from this analysis due to insufficient 
data. 

Respondents in 22 of the 25 core sample schools reported receiving support to build human capital 
such as professional learning (22 schools) or staffing support (3 schools) from an external support 
provider during Year 2 of SIG (2011-12). Among these 22 schools, 19 reported receiving support from 1 
to 3 providers, while the remaining 3 schools reported receiving support from 4 to 6 providers. Providers 
included colleges and universities, CMOs, EMOs, school turnaround organizations, curriculum 
developers, consultants from SEAs, and other organizations (such as local counties or advisory boards). 

Respondents in 13 of the 25 core sample schools reported receiving external support from a "SIG 
partner," an organization or individual that was considered central to the change process under SIG. 
Collectively, these schools identified a total of 17 SIG partners. Of the 15 SIG partners with sufficient 
data, 11 were described by respondents as being an appropriate fit for the school in terms of their 
expertise and experience. 

Overall, respondents described 12 of the 15 SIG partners with sufficient data in positive terms, 
reporting that the services, advice, or feedback they received contributed to the school's 
improvement efforts. Respondents described the remaining three SIG partners in mixed terms. In these 
instances, respondents identified ways in which the SIG partner's support was useful but also noted 
areas where the SIG partner's support was lacking. 

Conclusion to Part II (Chapter 7) 

The 25 core sample schools engaged in various efforts to build human capital during the first two years 
of SIG. However, such efforts are often undertaken in conjunction with other improvement actions. 

Thus, to gain a broader understanding of each school's overall efforts to improve capacity and the 
differences among them, we developed an aggregate measure. The aggregate measure included: efforts 
to build structures for distributed leadership; efforts to build structures to support teacher 
collaboration; efforts to build structures to support data use; the replacement of 50% of teachers; the 
addition of noninstructional staff; a purposeful approach to staffing and professional learning that is 
aligned to school goals and needs; hours of professional learning; the presence of designated staff to 
support SIG schools; and a SIG partner to support the school. Most indicators were scored according to a 
binary system, with 0 assigned to the negative category (e.g., "no evidence," "not identified") and 1 to 
the affirmative category (e.g., "identified"). 

In addition to variation in the specific activities that each school implemented, our aggregate measure 
suggests variation across the 25 core sample schools in the number of efforts to build human capital 
during the first two years of SIG. Aggregate ratings of school efforts to build human capital ranged from 
2.5 to 8.5 out of a maximum possible rating of 10, with 12 core sample schools scoring above 6, and 13 
scoring 6 or below. There was no apparent association between a school's aggregate measure of efforts 
to build human capital and its SIG intervention model, external context, urbanicity, or teachers' average 
years of experience. 

The most common approach to building human capital among the 25 core sample schools was adding 
noninstructional staff positions. All but one core sample school reported adding such staff in either Year 
1 or 2 of SIG, and 13 of these schools reported adding such staff in both years. 


XII 


Case Studies of Schools Receiving School Improvement Grants: Final Report 


The four core sample schools in which respondents described SIG as the primary impetus for change in 
Year 1 of SIG (see Le Floch et al., 2014) had a higher aggregate score of human-capital-building efforts 
than did schools in which SIG was not perceived to be an impetus for change. The four schools in 
which SIG was perceived as the primary impetus for change in Year 1 scored an average of 7.75 on our 
aggregate measure of human-capital-building efforts, compared with an average score of 5.67 among 
the 21 schools in which respondents did not perceive SIG as the impetus for change. 

Changes in Perceived Improvement and School-Level Capacity to Improve 
Student Learning (Chapter 8) 

SIG was intended to provide the impetus and means necessary for persistently low-performing schools 
to take dramatic and purposeful action toward improving school and student outcomes. To understand 
how school stakeholders perceived their schools to have changed over the three years of SIG, and the 
ways in which schools shifted in their organizational capacity, we explored the commonalities and 
distinguishing features of a subsample of 12 of the 25 core sample schools (collectively referred to as 
the core subsample), which had complete site visit and teacher survey data over the three years. 

Most teachers in 7 of the 12 core subsample schools reported on the Year 3 teacher survey that their 
school had changed in primarily positive ways throughout the course of SIG (see Exhibit ES.2). Across 
all 12 schools, the average percentage of teachers who reported overall positive improvement was 53 
percent, compared with 8 percent of teachers who reported negative changes overall. There was, 
however, variation across schools, with the percentage of teachers who reported that their school had 
changed in primarily positive ways ranging from 0 percent in one core subsample school to 97 percent in 
another. 

The quality of leadership in the 12 core subsample schools appeared to be related to teachers' 
perceptions of school improvement, based on data from the teacher survey and site visits. In Year 3 of 
SIG (2012-13), teachers in all 3 of the 12 core subsample schools whose principals were reported as 
exhibiting higher levels of teacher trust and overall leadership (including transformation, instructional, 
and strategic leadership) also reported the highest levels of perceived improvement over the three years 
of SIG (see Le Floch et al. [2014] for more detail on the analyses of principal leadership). Conversely, the 
four core subsample schools with the lowest aggregate ratings across our leadership measures were 
also the ones whose teachers reported the lowest levels of perceived improvement over the course of 
SIG. 

A school's level of organizational capacity in Year 3 of SIG (2012-13) appeared to be related to their 
organizational capacity in Year 1 (2010-11). All four core subsample schools that had relatively lower 
organizational capacity in Year 1 appeared to improve to moderate or higher capacity by Year 3. Among 
the four schools with moderate capacity in Year 1, three remained moderate by Year 3. Similarly, all four 
schools with higher levels of capacity in Year 1 maintained a high level of capacity by Year 3. 

Schools that appeared to make more effort to build human capital in Years 1 and 2 of SIG were more 
likely to improve their organizational capacity (or sustain their already higher capacity) than schools 
that appeared to make less effort to build human capital. Three of the seven core subsample schools 
that reported many efforts to build human capital improved their organizational capacity from Year 1 to 
Year 3. Among the five core subsample schools that reported fewer efforts to build human capital in 
Years 1 and 2 of SIG, one increased organizational capacity in Year 3 and three remained stable, while 
one decreased organizational capacity. 
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Exhibit ES.2. 

Overall Perceived Improvement During SIG, by Core Subsample School, 2010-13 
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Source: SST teacher survey, spring 2013. 

Notes: Includes 12 core subsample schools. All school names are pseudonyms. 


Sustainability of the Improvement Process (Chapter 9) 

Research suggests that sustainability entails maintaining, extending, and adapting improvement efforts 
in an unremitting reflective process (Jerald, 2005). Researchers have reported a relationship between 
school capacity and the ability to sustain reform efforts (Florian, 2000; Taylor, 2006). Though the SIG 
award is a temporary infusion of funds, the aim is ultimately to help set schools on a path of 
improvement that will continue beyond the course of the grant. We explored the relationship between 
organizational capacity and predicted sustainability among the 12 core subsample schools, as well as the 
schools' overall prospects for sustaining the reform efforts in the years beyond SIG. 

Core subsample schools that had higher levels of organizational capacity by Year 3 of SIG (2012-13) 
also had higher scores on the teacher survey scale measuring perceived sustainability (see Exhibit 
ES.3). 

Respondents most often cited human capital as a risk factor for sustaining reform efforts beyond SIG. 
In particular, respondents in 9 of the 12 core subsample schools expressed concern about losing staff; 
respondents in 2 of the 12 schools explicitly linked sustainability concerns to an impending change in 
school leadership. In 5 of the 9 schools where respondents identified the loss of staff as a threat to 
sustainability, respondents anticipated losing staff whose positions were funded through SIG, including 
instructional coaches, dropout prevention staff, and community or parent-community coordinators. 
Flowever, staff in 5 of these schools also believed that they would be able to retain at least some of the 
staff newly hired through SIG. Respondents in 4 of the 12 core subsample schools already knew that 
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they were going to experience a change in leadership in fall 2013, and respondents in 2 of those schools 
explicitly associated sustainability concerns with the impending loss of their current principal. 


Exhibit ES.3. 

Perceived Sustainability Index and Organizational Capacity, 2012-13 
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Source: Sustainability index: SST teacher survey, spring 2013. Organizational capacity index: Principal, instructional coach, and 
teacher interviews; teacher, leadership team, and community focus groups; SST teacher survey, spring 2013. 

Notes: Includes 12 core subsample schools. All school names are pseudonyms. 


Of the 12 core subsample schools, which we followed for all three years of SIG, 2 appeared to have 
strong prospects for sustainability, 6 schools appeared to have mixed prospects, and the remaining 4 
schools appeared to have weak prospects for sustainability, according to teacher survey responses 
and site visit data. At the two schools with strong prospects for sustainability, respondents expressed 
confidence that their schools had the resources to sustain and build upon their improvements. 

Schools' prospects for sustainability appear to be unrelated to several variables that might be 
hypothesized to predict sustainability, such as the size of the school's grant relative to baseline 
spending, enrollment, school level, or the type of SIG intervention model. 

Respondents in 2 of the 12 core subsample schools provided evidence of ownership of the 
improvement process by Year 3 of SIG (2012-13). In the remaining 10 schools, not enough data were 
collected to make an assessment of ownership. 

Conclusion 

The school change process is complex, and crafting policy that acknowledges this complexity while 
compelling change has challenged policymakers for decades. Low-performing schools are not blank 
slates, on which new interventions and individuals can be imposed and assumed to stimulate better 
outcomes for children. These new policies are inserted into a complex policy context, history, and set of 
assumptions about each school. Our study provides evidence that these persistently low-performing 
schools could potentially change, at least in the short term, after engaging in numerous efforts to build 
human capital. Sustaining these changes may require an equally great effort to retain any hard-won 
improvements. 
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Part I: Setting the Stage 
Chapter 1: Introduction 


The Study of School Turnaround (SST) examines the improvement process in a diverse, purposive 
sample of schools receiving School Improvement Grants (SIG) under Title I Section 1003(g) of the 
Elementary and Secondary Education Act (ESEA). SIG provides formula-based federal funds to states that 
then competitively award the funds to districts applying for SIG on behalf of their low-performing 
schools. SIG schools use the funds to implement improvement strategies. First authorized in 2001, SIG 
received relatively low levels of funding until the passage of the American Recovery and Reinvestment 
Act of 2009 (ARRA), which injected $3 billion into the program. This study focuses on schools that 
received SIG as part of the first cohort of grantees after ARRA to implement improvement strategies 
over three years, from 2010-11 to 2012-13. 

Policy Overview 

Congress introduced provisions to ESEA in 1988 to hold schools accountable for improving the 
performance of their students. The 1994 ESEA authorization (Improving America's Schools Act) tied 
these provisions to state-adopted standards in reading and mathematics, and introduced the notion of 
adequate yearly progress (AYP). The 2001 ESEA reauthorization (No Child Left Behind Act or NCLB) 
specified criteria for identifying low-performing schools and required actions and interventions intended 
to improve student outcomes in schools that failed to meet AYP. By 2008-09, 12,599 schools nationwide 
had been identified for improvement, corrective action, or restructuring under Title I of ESEA (Taylor et 
al., 2010). Of these, 5,017 schools were in restructuring status, meaning that they had failed to meet 
AYP for at least five years (Taylor et al., 2010). 

Although SIG was first created with the 2001 ESEA reauthorization under Title I Section 1003(g), new 
guidance for SIG issued in 2010 by the U.S. Department of Education (ED) sought to strengthen the 
program in two ways. 8 First, SIG sought to further prioritize the nation's worst schools by concentrating 
SIG resources in those schools that were performing especially poorly for a number of years. Although 
NCLB aimed to identify low-performing schools, the AYP criteria did not focus only on schools with the 
lowest overall performance. For example, an NCLB-identified school may have missed AYP targets for 
one subgroup or a single subject area, rather than for all subgroups and both English language arts (ELA) 
and mathematics. Under the 2010 SIG guidance, schools were now classified into three eligibility tiers 
based on their overall performance. ED required states to prioritize SIG awards to Tier I and Tier II 
schools, which had the worst achievement levels among schools eligible for SIG: 

• Tier I includes any Title I school in improvement, corrective action, or restructuring that (1) is 
among the lowest achieving 5 percent of the schools in these school improvement categories in 
the state or (2) is a high school that has had a graduation rate below 60 percent for a number of 
years. States have the option of identifying Title l-eligible 9 elementary schools that (1) are no 


8 For more information on SIG, see the U.S. Department of Education's webpage on SIG legislation, regulation, and 
guidance (http://www2.ed.gov/programs/sif/legislation.html). 

9 Title l-eligible schools refer to those schools that do not receive Title I funds but may meet the criteria for 
obtaining the funds. 
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higher achieving than the highest achieving school in Tier I and (2) have not made AYP for at 
least two consecutive years or are in the state's lowest quintile based on proficiency rates. 

• Tier II includes any secondary school that is eligible for but does not receive Title I, Part A funds 
and (1) is among the lowest achieving 5 percent of such secondary schools in the state, or (2) 
has had a graduation rate below 60 percent for a number of years. (States also may identify Title 
l-eligible secondary schools that (1) are no higher achieving than the highest achieving school 
identified as a persistently lowest achieving school in Tier II or have had a graduation rate of less 
than 60 percent during a number of years, and (2) have not made AYP for at least two 
consecutive years or are in the state's lowest quintile based on proficiency rates.) 

• Tier III includes the remaining Title I schools in improvement, corrective action, or restructuring 
that are not Tier I schools. (States have the option of identifying as Tier III schools Title l-eligible 
schools that (1) do not meet the requirements to be in Tier I or Tier II, and (2) have not made 
AYP for at least two consecutive years or are in the state's lowest quintile based on proficiency 
rates.) 

Second, SIG aimed to catalyze more aggressive improvement efforts by requiring grantee schools to 
adopt one of four specific intervention models. Although NCLB delineated a set of corrective actions, 
identified schools tended not to adopt the most aggressive ones. For example, schools that failed to 
meet AYP targets for at least five years (and were thus in restructuring status) had five options: (1) 
replace all or most of the school staff, (2) allow the state to take over the school, (3) reopen the school 
as a public charter school, (4) contract with a private entity to manage the school, or (5) implement "any 
other major restructuring of the school's governance arrangement that makes fundamental reforms, 
such as significant changes in the school's staffing and governance, to improve student academic 
achievement in the school and that has substantial promise of enabling the school to make AYP as 
defined in the State plan" (NCLB, 2001). The National Longitudinal Study of NCLB found that in 2006-07, 
22 percent of the schools in restructuring status had implemented one of the first four more stringent 
options (Taylor et al., 2010). An earlier U.S. Government Accountability Office (2007) report similarly 
found that about 40 percent of the schools in restructuring had not implemented any of the five 
options. Under the 2010 SIG guidance, Tier I and Tier II schools that received SIG were required to 
implement one of four intervention models. The key requirements for each model are as follows: 

1. Turnaround model: Replace the principal and no less than 50 percent of the staff, introduce 
significant instructional reforms, increase learning time, and provide the school sufficient 
operational flexibility (i.e., staffing, time, and budgeting) and support (e.g., ongoing, intensive 
technical assistance and related support). 

2. Restart model: Reopen the school under the management of a charter school operator, a 
charter management organization, or an education management organization (must enroll, 
within the grades served, any former student who wants to attend the school). 

3. School closure: Close the school and reassign students to higher achieving schools. 

4. Transformation model: Replace the principal, develop a teacher- and leader-evaluation system 
that takes student progress into account, introduce significant instructional reforms, increase 
learning time, and provide the school sufficient operational flexibility and support. 

Each state's SIG allotment is determined by a formula based on Title I allocations. In turn, state 
education agencies (SEAs) award funds to local education agencies (LEAs) with eligible schools, based on 
a competitive application process. Awards must be made based on the eligibility tiers established by the 
federal guidelines and in accordance with SEA determinations of LEA capacity and commitment to 
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support school turnaround. SEAs were able to award LEAs up to $2 million annually over three years for 
each qualified SIG school. 10 

With passage of ARRA in 2009, the SIG program received an additional $3 billion to go with its regular 
appropriations. Between fiscal years 2009 and 2012, SIG infused over $5 billion in total to the nation's 
persistently lowest-achieving schools (see Exhibit 1.1). During this time period, SEAs held competitions 
for two cohorts of LEAs. Cohort I, the focus of this study, included districts and schools that received SIG 
to implement reforms beginning in the 2010-11 school year and continuing through the 2012-13 school 
year. Cohort II included districts and schools that received SIG to implement reforms beginning in the 
2011-12 school year and continuing through the 2013-14 school year. Box 1.1 describes the 
demographic characteristics, models, and funding levels of the first cohort of SIG schools nationwide. 


Exhibit 1.1. 

Annual Federal Appropriations for SIG From 2007 to 2012 


Fiscal Year 

Amount 

Funding Recipients 

2007 

$125,000,000 

Pre-ARRA grantees 

2008 

$491,265 

Pre-ARRA grantees 

2009 

$3,546,000,000* 

Cohort 1 grantees: 

Years 1, 2, and 3 of implementation (2010-11 to 2012-13) 



Cohort II grantees: 

2010 

$546,000,000 

Year 1 of implementation (2011-12) 

2011 

$535,000,000 

Year 2 of implementation (2012-13) 

2012 

$535,000,000 

Year 3 of implementation (2013-14) 


Source: U.S. Department of Education School Improvement Grants website: http://www2.ed.gov/programs/sif/funding.html. 
Originally published in Hurlburt, Therriault, & Le Floch (2012). 

Notes: Each grantee school typically receives an award to implement reforms for three years. States with fiscal year (FY) 2009 
carryover funds (i.e., unused funds from their Cohort I competition) were allowed to use these funds to make similar three-year 
awards in their Cohort II competition. Thus, Cohort II grantees also include schools awarded SIG through carryover funds from FY 2009. 
■"Includes the regular appropriation of $546 million from Title I Section 1003(g), as well as $3 billion from ARRA. 


10 The Consolidated Appropriations Act (2010) raised the maximum funding amount for a participating school from 
$500,000 to $2,000,000 per year. 
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Box 1.1. Key Findings From the Cohort I Baseline Report 

Number and characteristics of SIG schools. Among the 49 states (and the District of Columbia) with 
available data, 1,228 schools received SIG. SIG schools were more likely to be high poverty (68 percent 
of students in SIG schools were eligible for free or reduced-price lunch compared with 45 percent of 
students nationwide), high minority (73 percent of students in SIG schools are students of color 
compared with 45 percent of students nationwide), and located in urban areas (53 percent of SIG 
schools were in large or middle-sized cities compared with 26 percent of schools nationwide). They were 
also more likely to be high schools (40 percent of SIG schools were high schools compared with 21 
percent nationwide). 

Intervention models adopted. Among Tier I and II SIG schools, 74 percent adopted the transformation 
model, 20 percent adopted the turnaround model, 4 percent adopted the restart model, and 2 percent 
adopted the school closure model. In 16 states, the transformation model was the only intervention 
model adopted by Tier I and II SIG schools. 

Absolute SIG award levels. SIG awards varied in absolute terms by tier, state, and school level. The 
average three-year award was $2.54 million for Tier I and II schools, and $520,000 for Tier III schools. 

The average three-year award for Tier I and II schools ranged from $620,000 in Vermont to $4.63 million 
in Illinois. The average three-year award was $2.37 million for high schools and $1.37 million for 
elementary schools. 

Relative SIG award levels. SIG awards varied by state in relative terms. Tier I and II SIG awards in 4 
states averaged 6 percent or less of what was spent overall in 2009-10 per pupil. (The average baseline 
spending among these states ranged from $10,700 to $13,400 per pupil.) In contrast, Tier I and II SIG 
awards in 11 states averaged 30 percent or more of what was spent overall in 2009-10 per pupil. (The 
average baseline spending in these states ranged from $6,400 to $23,500 per pupil.) 

Source: Hurlburt et al. (2011). 


Study Purpose and Conceptual Approach 

The purpose of the Study of School Turnaround was to examine how a variety of low-performing schools 
approached the improvement process during the three years in which they received SIG, and how SIG 
contributed to this process. In particular, we conducted case studies in a small but diverse sample of 
Cohort I SIG schools to examine the characteristics of the schools, the decisions and strategies that the 
schools undertook (and why), and the challenges they faced as they attempted to dramatically improve 
school performance. The study is based primarily on interviews and surveys, which, by nature, are self- 
reported. These data were not audited or validated with external sources. SST does not examine the 
impact of SIG on student achievement outcomes because the case study approach is not well-suited to 
investigate these questions rigorously, and, because of the limited sample size, it is not designed to 
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document the practices of all, or even a representative sample of, SIG schools nationwide. 11 This is the 
final report for this study. 12 

Conceptual Framework 

We developed a conceptual framework to help identify and define the main constructs, guide the 
development of data collection instruments, and focus the analyses of data (see Exhibit 1.2). This 
framework is based on the SIG program requirements and the research literature on organizational 
change processes, policy implementation, and effective schools. Prior studies on improving low- 
performing schools have hypothesized relationships among a range of programmatic and organizational 
variables on the one hand, and improvements in teaching and learning on the other (Bryk et al., 2010; 
Herman et al., 2008; McLaughlin & Talbert, 2006). However, it is important to caution that the evidence 
supporting these relationships is mixed, with some relationships having stronger empirical support than 
others. Below we highlight how the conceptual framework reflects the study's central assumptions, 
which are drawn from organizational theory and the literature on school change. A complete review of 
the evidence base for all of the constructs and hypothesized relationships in the framework is beyond 
the scope of this report (see Le Floch et al. [2014] for a more complete discussion of the literature that 
supports our conceptual framework). 

Internal Processes and Factors 

Assumption: Improvement is primarily a process within the school and is primarily affected by internal 
factors such as the existing practices and reform history in the school, staff definitions of the 
performance problem, the quality and focus of site leadership, knowledge and skills of instructional 
staff, and the internal norms governing staff interactions and practice (O'Day, 2002). As is the case in 
other complex organizations, changes or interventions in one or more areas within a school will 
influence changes in other areas of organizational functioning (Axelrod & Cohen, 1999). 

The box labeled Improvement Activity in SIG Schools (see Exhibit 1.2) reflects the centrality of the school 
and the factors within the school that are hypothesized to shape the improvement process. Within this 
box are aspects of the change process in these schools. 

First is the definition of the performance problem. Some researchers have posited that individual and 
collective definitions of the performance problem, implicit and explicit theories of causality and change, 
and interpretations of strategies and their apparent results can shape the improvement strategies and 
actions of the school actors, as well as those of the school district and support providers (Spillane, 

Reiser, & Gomez, 2006). 


11 IES is conducting an evaluation of SIG (Implementation and Impact Evaluation of Race to the Top and School 
Improvement Grants) to examine the impact of SIG on school and student outcomes using a larger sample of 
schools and methods more appropriate for estimating impacts and how they are correlated with various broader 
measures of implementation. These findings are anticipated by the end of 2016. 

12 See Boyle et al. (2014), Golden et al. (2014), Le Floch et al. (2014), and Rosenberg et al. (2014) for additional 
findings from this study. 
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Exhibit 1.2. 

Conceptual Framework 
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Contextual Influences (all levels): past reform efforts; fiscal resources; 
implementation of otherfederal programs (Title I, II, III; RacetotheTop); 
other relevant state and local policies; stakeholder relations and political 
climate; external school environment (e.g., safety, community support) 

State Implementation of SIG 
State SIG policies: SIG eligibility, selection process for school districts, 
support and guidance 

Dimensions of implementation: coherence, specificity, authority 


District Implementation of SIG 
Definition(s) of the performance problem: needs assessment, staff 
j interpretation of performance data, staff interpretation of SIG, goal 
: setting 

* Strategies and actions in key improvement domains (descriptive): 

j human capital management (e.g., teacher evaluation system, 

: professional learning, principal replacement, support); technical core 
(e.g., curriculum and instruction, data use); financial resource allocation 
Dimensions of implementation: consistency, divergence from prior 
practice, specificity, authority, sustainability 


i f. 


Improvement Activity in SIG Schools 
Definition(s) of the performance problem: needs assessment, 
staff interpretation of performance data, staff interpretation of 
SIG, goal setting 

Leadership for change: transformational, instructional, strategic, 
distributed 

Strategies and actions in key improvement domains: 

• Human capital management: staff recruitment and 
replacement, professional learning, evaluation and support 

• Technical core: curriculum and pedagogy, use of time, data 
use, student supports 

• Conditions to support teaching and learning: parent and 
community, school climate, financial resource allocation 

Dimensions of implementation: consistency, divergence from 
prior practice, breadth, depth, buy-in, sustainability 




I t 


Leading Indicators of Improvement 

Teacher collaboration, safe and orderly climate, teachers' sense of 
trust, clear and shared goals, quality of leadership, use of data for 
instructional decisions, programmatic coherence, locus of 
responsibility, staff stability, staff knowledge and skills, staff 
expectations of students, student and teacher engagement 

* - E 


External 

Partners 
Key improvement 
domains: 

professional 
learning, student 
supports, use of 
time, parent and 
community 
Dimensions of 
implementation: 
consistency, fit, 
intensity, stability, 
accountability 


Student outcomes 


Student outcomes in 


Student outcomes in 

before SIG 


Year 1 


Years 2-3 
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Second is leadership for change. Principal leadership reflects a complex set of skills such as 
communicating a vision, knowledge of content and pedagogical techniques, ability to build a committed 
staff, and knowledge of how to use data for decision making. These skills appear to be associated with 
rapid school improvement, as indicated by case study research (Herman et al., 2008; Usdan, McCloud, & 
Podmostko, 2000). SIG emphasizes the importance of principal leadership by requiring that schools 
implementing the turnaround or transformation models replace any principals who have been in their 
positions for more than two years prior to receiving SIG. 

Third are the specific improvement strategies and actions that schools can adopt. Based on SIG guidance 
and the literature on school improvement, our framework focuses on strategies and actions related to 
the following three areas: 

• Human capital. The important role of teacher quality in student learning is well documented, as 
theory and correlational research have related school success to staff capacity. 13 Staff 
replacement, professional learning, and performance management systems aim to improve staff 
knowledge and skills, and are common approaches to building human capital in low-performing 
schools. SIG schools implementing the turnaround model are required to not only replace the 
principal but also at least 50 percent of their teachers. SIG schools implementing the 
transformation model are required to adopt teacher evaluation systems that take student 
achievement into account, in addition to replacing the principal. 

• Technical core of instruction (what and how students learn). At the heart of most analyses of 
school improvement is the assumption that what occurs in the classroom helps determine what 
students learn. SIG requires schools implementing the turnaround and transformation models to 
adopt "significant instructional reforms" and to increase learning time for students (U.S. 
Department of Education, 2010). Researchers have studied the effects on student outcomes of 
increasing learning time, and numerous possible changes in curriculum and pedagogy or in the 
academic and nonacademic supports for students. (The What Works Clearinghouse provides 
examples of specific mathematics, science, and literacy curricula that have evidence of being 
able to produce achievement gains; for research on learning time, see Mass [2020], Time and 
Learning, a Brief Review of the Research ; for research on nonacademic supports, see Osher & 
Kendziora [2010] and Osher et al. [2014].) 

• School conditions that support teaching and learning. Improving school climate, engaging 
parents and community members, or allocating increased resources all have been hypothesized 
to be related to student outcomes. Low-performing schools often tackle conditions such as 
school climate and parent or community support prior to making changes in the technical core 
of instruction (Bryk et al., 2010; Herman et al., 2008). 

External Influences on School Change 

Assumption: Change in schools is also influenced by external factors and conditions. Schools are 
situated in larger complex systems— states and districts— whose strategies (as is the case with 
schools) reflect varying theories of action, including both conceptions of the underlying problems and 
assumptions about how the chosen strategies will address those problems (Argyris & Schon, 1974; 

City et al., 2009; O'Day & Bitter, 2003; Smith & O'Day, 1991; Weiss, 1995). 


13 See Raudenbush (2014) for an analysis of recent studies: 
http://www.carnegieknowledgenetwork.org/briefs/long-term-impacts. 
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The white boxes in Exhibit 1.2 highlight the influence of other system actors— states, districts, and 
external providers— on what happens in a school. As a federal program, SIG funds flow through states to 
districts and ultimately to schools. States and districts make decisions that determine which schools 
receive SIG, how much funding they receive, and which models they will adopt. The SIG guidelines also 
encourage funded schools to partner in some capacity with an external support provider, such as 
educational management organizations, charter management organizations, outside vendors, and 
nonprofit organizations, with the hope that these partners will positively influence the implementation 
of improvement strategies. 

The gray area surrounding the boxes in Exhibit 1.2 represents the contextual conditions that can 
influence SIG implementation at all levels. The actions that SIG schools take may be related to non-SIG 
policies and programs at the federal, state, and local levels; the political and fiscal climate; and the 
demographics and needs of the communities and families served. For example, SIG is targeted to low- 
performing schools, many of which are located in low-income neighborhoods. The relationship between 
neighborhood poverty and student outcomes is well documented (Duncan & Raudenbush, 1999; Pebley 
& Sastry, 2004). Neighborhood characteristics such as economic and social flux, high resident turnover, 
and a large proportion of single-parent families— as well as neighborhoods where crime and drugs are 
present— all represent risk factors for youth development (Flann & Borek, 2001) and can be reflected in 
unsafe schools (Neiman & DeVoe, 2009). 

Dimensions of Implementation 

Assumption: Improvement actions comprise both what is done and how (or how well) it is done. 
Adoption of improvement activities or strategies is not enough. Their success in changing practice 
depends on the quality of implementation. Implementation and change are influenced by the 
individual and collective beliefs, motivation, and knowledge/skills of school personnel (McLaughlin, 
2005; Spillane, Reiser, & Reimer, 2002). 

Reflecting the role of implementation in school improvement, the box in the conceptual framework 
labeled Improvement Activity in SIG Schools includes both descriptive characteristics of the change 
strategies (i.e., actions that schools undertake to improve human capital, technical core, or conditions 
that support teaching and learning) and analytic dimensions associated with their implementation (i.e., 
features of these change strategies, such as their consistency with one another or how they diverge 
from prior practice. Dimensions of implementation also appear in the white boxes related to state and 
district implementation of SIG and the actions of external partners. 

Correlational studies and case study research indicate that the level and quality of implementation are 
associated with the prospects for school turnaround (Newmann & Associates, 1996; Newmann et al., 
2001) and have identified several variables that relate to the implementation of successful improvement 
efforts. These variables include the divergence from prior practice (Flerman et al., 2008; Rhim et al., 
2007), and teacher "buy-in" (Bailey, 2000; Bodilly, Purnell, Ramsey, & Keith, 1996; Datnow, 2000). 
Researchers have proposed varying hypotheses regarding how some of these implementation variables 
influence school change. For example, divergence from prior practice may make implementation of a 
reform more difficult because the reform may conflict with entrenched patterns of behavior; on the 
other hand, efforts that diverge from prior practice may have the best prospects for "shaking up" 
entrenched routines and expectations (see Rhim et al., 2007; Waters, Marzano, & McNulty, 2003). 

In both theoretical and empirical studies, researchers have described the complexity of implementing 
school improvement efforts. Implementation takes shape as policies and practices are interpreted and 
acted on across multiple levels of the education system, across individuals within these levels, and over 
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time as conditions change and as individuals and organizations interpret the results and modify practices 
(Honig, 2006; McLaughlin, 1987; Spillane, Reiser, & Reimer, 2002). 

Leading Indicators and Organizational Capacity 

Assumption: While the ultimate goal of the improvement process is to increase student achievement, 
many improvement actions that schools take are designed to operate on less direct influences on 
student learning, such as teacher knowledge and skills, principal leadership, or school climate. Taken 
together, these indirect influences are often referred to as schools' organizational capacity for high 
performance and improvement. Studies of low-performing schools have often noted their general lack 
of organizational capacity (Mintrop & Trujillo, 2005; O'Day, Goertz, & Floden, 1995) and cite low 
capacity as an important contributor to persistent low achievement. By contrast, studies of low- 
performing schools that perform better than expected demonstrate a correlation between these same 
characteristics and consistently higher performance (Hallinger & Heck, 2010; Harris, 2002). Although 
these studies have been mainly correlational in nature, they suggest that indicators of school capacity 
are both malleable and potentially predictive of subsequent improvements in student outcomes. 
Capacity is therefore both a target of improvement activity and a leading indicator of improvement. 

The box in the conceptual model labeled Leading Indicators of Improvement lists aspects of a school's 
capacity that researchers hypothesize set the school-level conditions that could foreshadow student 
learning outcomes. For example, staff knowledge and skills is an indicator of the human capital aspect of 
capacity. Researchers have documented the relationships between the knowledge, skills, dispositions of 
teachers and leaders— human capital— and student outcomes (Cohen & Ball, 1999; Rockoff, 2004), and 
approaches to improving low-performing schools have tended to focus on building human capital 
(Massell, 2000; McLaughlin & Talbert, 2006). 

Exhibit 1.3 describes the subset of capacity indicators that we were able to measure in this study, and 
specifies some of the literature that supports their hypothesized relationships to student outcomes. As 
is clear from the literature, human capital alone does not constitute school capacity. Mutual 
understanding and norms that grow out of relationships among individuals in the school; coherent 
instructional frameworks that are incorporated into practice; and proper use of resources, which can 
include curriculum materials or instructional time, all are aspects of the school's capacity as an 
organization (Beaver & Weinbaum, 2012; Bryk et al., 2010)). Organizational capacity also includes vision 
and leadership, access to knowledge, and organizational structures and management (King & Bouchard, 
2011; O'Day, Goertz, & Floden, 1995). 
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Exhibit 1.3. 

Leading indicators of Organizational Capacity 


Leading 

Indicator 

Definition 

Supporting Literature* 

Teacher 

Collaboration 

Teacher collaboration is characterized by 
mutual assistance and support within the 
school context (O'Day, Goertz, & Floden, 
1995). Often described in the literature as 
either same-subject teachers "identifying a 
common curriculum, developing common 
assessments aligned to that curriculum, and 
then analyzing common assessment data to 
make instructional changes" (DuFour, 

2004b, p. 3) or as teachers of the same 
students but of different subjects working 
together (Erb & Doda, 1989; Rottier, 2001). 

Some studies have found a positive correlation between 
teacher collaboration and student achievement (Goddard, 
Goddard, & Tschannen-Moran, 2007; Vescio, Ross, & 

Adams, 2008). Teacher collaboration and cooperation 
facilitate improved teacher morale and motivation 
through the sharing of ideas and practices (Corcoran & 
Goertz, 1995). This mutual assistance and support, or the 
"receptivity" of colleagues, plays a role in teachers' daily 
practice (O'Day, Goertz, & Floden, 1995). 

Safe and 

Orderly 

Climate 

A safe and orderly climate is an 
environment in which students "have a 
sense of being physically and psychologically 
safe in their school. There are few 
disruptions due to disciplinary problems, 
and those that occur are handled firmly and 
fairly" (Consortium on Chicago School 
Research, 2004, Student-Centered Learning 
Climate section). 

A safe school environment characterizes schools that 
have beaten the odds (Bryk et al., 2010; Herman et al., 

2008; Johnson & Asera, 1999; U.S. Department of 

Education, 2010). "Prevailing research suggests that 
students' feelings of safety at school, and problems with 
peer relationships and bullying, are influenced by a broad 
array of factors, including students' own attributes, 
attributes of their schools, adults with whom students 
interact, families, neighborhoods, and the broader 
society" (Steinberg, Allensworth, & Johnson, 2011, p. 8). 

Teachers' 

Sense of Trust 

Teachers' sense of trust is referred to as the 
extent to which teachers feel they have 
mutual respect for each other, for those 
who lead school improvement efforts, and 
for those who are experts at their craft 
(Consortium on Chicago School Research, 
2004). 

Based on correlational analyses of survey data, Sebring 
and Bryk (2000) found that "in schools that are improving, 
where trust and cooperative adult efforts are strong, 
students report that they feel safe, sense that teachers 
care about them, and experience greater academic 
challenge. In contrast, in schools with flat or declining test 
scores, teachers are more likely to state that they do not 
trust one another" (p. 5). 

Clear and 

Shared Goals 

Schools in which goals are clear and shared 
among staff are characterized by a unity of 
purpose, explicit expectations, and shared 
values for student learning and success 
(Newmann et al., 2001; Purkey & Smith, 

1983). 

Studies of schools with higher-than-expected 
achievement found that establishment of a clearly 
defined purpose enables a school to "direct its resources 
and shape its functioning toward the realization of those 
goals" (Purkey & Smith, 1983, p. 445) and helps to reduce 
student alienation (Newmann, 1981). Research about 
organizations other than schools has found that shared 
values among colleagues is related to one's personal 
sense of investment in the organization and facilitates 
cooperation in the workplace (Cable & DeRue, 2002; 
Watrous, Huffman, & Pritchard, 2006). 

Quality of 
Leadership 

A school principal demonstrating quality 
leadership in an improving school is "more 
likely to be an instructional leader, more 
assertive in his/her institutional leadership 
role, more of a disciplinarian, and. ..assumes 
responsibility for the evaluation of the 
achievement of basic objectives" (Edmonds, 
1979, p. 18). For the purposes of the 
analyses in this report, three dimensions of 
leadership are addressed: transformational 
leadership, instructional leadership, and 
strategic leadership. 

Case studies of successful turnaround schools consistently 
point to the role of the principal in turnaround efforts 
(Edmonds, 1979; Herman et al., 2008; Purkey & Smith, 
1983). One meta-analysis of 70 studies of principal 
leadership found a positive correlation between principal 
leadership (as measured by teacher perceptions) and 
student achievement (Waters, Marzano, & McNulty, 

2003). 
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Exhibit 1.3. 

Leading Indicators of Organizational Capacity (continued from previous page) 


Leading 

Indicator 

Definition 

Supporting Literature* 

Use of Data for 

Instructional 

Decisions 

The use of data for instructional decisions is 
characterized as the monitoring of student 
learning and frequent and transparent use 
of student outcome data to guide 
instructional decisions (Coburn & Beuschel, 
2012; Coburn & Turner, 2011a; Coburn & 
Turner, 2011b). 

Using data to modify curricular and teaching strategies is 
a common feature of turnaround schools (Herman et al., 
2008). Some studies have found that data can help 
teachers fine-tune their practices and catch learning 
problems before they become intractable, in some cases 
diminishing referrals to special education programs 
(Marston et al., 2003; McNamara, 1998; Reschly & 
Starkweather, 1997; Sornson, Frost, & Burns, 2005). 

Programmatic 

Coherence 

Programmatic coherence is measured by the 
degree to which the policies of a school 
reflect consistent goals, the strategies 
employed are clearly designed to foster 
achievement of these goals, and barriers 
and detractors from the goals and strategies 
are systematically removed (Honig & Hatch, 
2004; Newmann et al., 2001). 

Correlational and case studies of schools implementing 
whole-school reforms found that school staff have 
difficulty implementing multiple, unrelated interventions 
(Berends, 2000; Berends, Bodilly, & Kirby, 2002) and that 
isolated interventions that are not aligned with other 
school or district objectives are less likely to achieve 
desired outcomes than interventions that are closely 
aligned with existing improvement efforts (Datnow et al., 
2006). 

Locus of 
Responsibility 

Locus of responsibility is characterized by 
the way in which school respondents 
attributed the performance problem in their 
school to factors within their control (i.e., 
internal causes) or outside their control (i.e., 
external causes). 

Reviews of research have found that schools in which 
teachers exhibit high levels of collective efficacy and take 
ownership for the challenges facing their schools are 
more likely to improve student outcomes (Bandura, 1993; 
Goddard, 2001; Goddard, Hoy, & Hoy, 2000; Goddard, 

Hoy, & Hoy, 2004). 


*The literature referenced in this exhibit includes conceptual as well as empirical work (mainly correlational and case study 
research). The exhibit is not a full review of the research about the leading indicators included in our conceptual framework. As 
with the conceptual framework as a whole, the leading indicators reflect the variables that researchers and educators 
hypothesize are related to student outcomes, rather than variables that have been conclusively determined to be causally 
related to student outcomes. 


To sum up, our conceptual framework depicts schools as complex social systems in flux that have a 
variety of internal stakeholders and prior histories of reform. The characteristics of schools and the 
various improvement strategies they employ interact and overlap. Schools also are situated in a variety 
of district, state, and community contexts. Implementation of any program, such as SIG, depends on 
how school-level actors interpret the performance problems of their schools, the approaches they take 
to address these problems, and conditions schools face. Increasing the school's capacity— both human 
and organizational— to improve student outcomes in the face of such complexity is thought to be a 
particularly difficult challenge facing many low-performing schools (Beaver & Weinbaum, 2012). 

Report Overview 

This report focuses on a small sample of SIG schools over the three years of the program, from 2010-11 
to 2012-13. It presents findings from the study's 25 core sample schools, which were the focus of data 
collection in spring 2011 and spring 2012, and a subsample of 12 of the 25 schools (collectively referred 
to as the core subsample), which were selected for data collection in spring 2013 and are the focus of 
analyses looking across the three years of SIG. 
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The findings in this report build upon the SST Year 1 report (Le Floch et al., 2014), which looked at the 
change process in the 25 core sample schools through the first year of SIG implementation (2010-11). 
Box 1.2 describes the key findings from the Year 1 report. 

The Year 1 data pointed to the diversity among the SIG schools in terms of neighborhood context, fiscal 
context, and principal leadership, as well as in the ways in which respondents defined the performance 
problem. In addition, the core sample schools engaged in a wide range of improvement actions and uses 
of SIG funds— not all schools had a large infusion of funds, and the SIG models did not necessarily 
dictate what actions the schools initiated. Given these variations, it is not surprising that at the end of 
the first year of SIG implementation, some schools appeared to be better positioned to improve student 
outcomes. 


Box 1.2. Key Findings From the Year 1 Report 

• Context and the Performance Problems in SIG Schools. Although all were low-performing, core 
sample schools differed in their community and fiscal contexts, their performance and reform 
histories, and their interpretations of the causes of— and potential solutions for— their 
performance problems. 

• Leadership for Change. Approaches to leadership varied across the set of core sample schools 
with most principals exhibiting a mix of leadership qualities. The most frequently reported 
leadership approach among the core sample schools was transformational leadership, referring 
to principals who can develop leaders and motivate and engage their staff behind a strong 
organizational vision. Although respondents in the majority of schools reported some 
improvement in 2010-11, schools in which respondents described the improvements in the 
greatest number of areas also had higher levels of principal strategic leadership (referring to 
principals who are able to formulate a strategy for school improvement and translate that 
strategy into concrete priorities and specific actions) and were more likely to have experienced a 
disruption from past practices. 

• Improvement Actions in SIG Core Sample Schools. For most of the core sample schools, 
respondents did not perceive SIG as the primary impetus for the change strategies that had 
been adopted. In 19 of these schools, the improvement strategies and actions implemented 
during the first year of the grant (2010-11) were reportedly a continuation of activities or plans 
that predated SIG. 

• SIG and the Change Process. At the time of data collection, 7 of the 25 core sample schools had 
experienced a visible disruption from past practice; the remaining schools appeared to be 
following a more incremental approach to improvement. 

• Leading Indicators of Change. Overall, core sample schools with the lowest levels of 
organizational capacity in 2010-11 were those in which teachers reported having fewer 
resources, the SIG award represented a larger percentage of the prior year's per-pupil 
expenditures, and respondents perceived the SIG award as a catalyst for change. 

Source: Le Floch et al. (2014). 


The Year 1 findings provided a baseline for exploring the change process in subsequent years. With data 
collected over the succeeding two years of SIG (2011-12 and 2012-13), we sought to document: (1) the 
schools' efforts to build human capital to address their challenges (the focus of data collection in Year 2) 
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and (2) the extent to which these efforts to build human capital yielded changes in organizational 
capacities to improve student learning, and the perceived sustainability of these improvement efforts 
(the focus of data collection in Year 3). This report, which is based on data from all three years of the 
study, is organized into three parts: 

Part I: Setting the Stage (Chapters 1-2). Chapter 1 introduces the SIG program and the study's 
conceptual framework. Chapter 2 describes the study design, timeline, sampling procedures, and data 
collection and analytic approaches. 

Part II: Building Human Capital (Chapters 3-7). The knowledge and skills of the school leadership and 
instructional staff provide the foundation and expected focus for much of the improvement efforts in 
SIG schools. The second part of this report examines the strategies that the 25 core sample schools 
employed to build their human capital. We focus on two general approaches or leverage points for 
improving the knowledge and skills of school staff: (1) change the composition of the school staff 
through replacement, recruitment, and retention strategies; and (2) change the capabilities and 
strategies of the existing staff through professional learning. We consider both the internal strategies in 
these two areas and the relevant influences and supports from external providers and school districts. 

Part III: Change and Sustainability (Chapters 8-9). SIG is intended to support rapid and dramatic 
improvement in the nation's persistently lowest-achieving schools. The chapters in the final section of 
this report focus on 12 of the 25 core sample schools (collectively referred to as the core subsample), 
examining school personnel's perceptions of improvement and changes in school capacity at the end of 
the three years of SIG, the role of SIG in those changes, and the perceived sustainability of the schools' 
improvement efforts at the conclusion of the grant. 

Because the SST was an exploratory study, the aim was to examine and generate hypotheses that might 
be explored in future research. Thus, the report ends with hypotheses and questions, setting the stage 
for future work to better understand the improvement process in the nation's persistently lowest- 
achieving schools. 
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Chapter 2: Study Design 


This chapter summarizes the study's design, including how we selected our sample of case study 
schools, when and what data we collected from these schools, and how we analyzed the data. 

We initially selected a base sample of 60 schools from the set of schools awarded SIG in the summer of 
2010 (Cohort I), which we then used to select various smaller samples for different parts of the study. 
The core sample consisted of 25 schools, which we visited during the 2010-11 and 2011-12 school 
years. For the 2012-13 school year, we selected a core subsample of 12 schools to visit from the 25 core 
sample schools. Thus, we followed 13 of the 25 core sample schools for two years (2010-11 and 2011- 
12) and the remaining 12 core sample schools for three years (2010-11 through 2012-13). These 25 
schools were subject to the most intensive data collection in this study, including interviews and focus 
groups with a range of respondents and teacher surveys. 

In addition, we selected two special topic samples from the base sample to more closely examine 
schools with large English language learner (ELL) populations and schools in rural settings: the ELL 
sample included 11 schools with a high concentration of ELLs, and the rural sample included 9 rural 
schools. We conducted site visits at the ELL sample schools during fall 2011 and 2012, and at the rural 
sample schools during spring 2012. There was some overlap among the various samples. For example, of 
the 11 schools in the ELL sample, 5 were also in the core sample. Of the 9 schools in the rural sample, 4 
were also in the core sample. One school was in both the rural and ELL samples, while one school was in 
the core, ELL, and rural samples. 

Exhibit 2.1 summarizes the various study samples and associated data collection activities and timelines. 
The rest of this chapter and report focuses on the core sample and core subsample schools. For more 
details on the ELL and rural samples, see the special topic evaluation briefs (Boyle et al., 2014; Golden et 
al., 2014; Rosenberg et al., 2014). 


Exhibit 2.1. 

Study Samples and Data Collection Timeline 


( 




Base Sample: 60 schools 

State interviews: Spring 2011 and 2012 


Core Sample: 25 schools 

- Site visits: Spring 2011 and 2012 

- Telephone or in-person interviews with key staff: Fall 2011 

- Teacher survey: Spring 2011 and 2012 

- Teacher survey supplement: Fall 2011 


I 


Core Subsample: 12 schools (in addition to data collection for all core sample 
schools) 

- Site visits: Spring 2013 

- Telephone or in-person interviews with key staff: Fall 2012 

- Teacher survey: Spring 2013 

- Teacher survey supplement: Fall 2012 


V. 

ELL Sample: 11 schools (5 overlap with core) 
-Site visits: Fall 2011 and 2012 

- Teacher survey: Spring 2011 and 2012 

- ELL teacher survey supplement: Fall 2011 and 
2012 


\ 


Rural Sample: 9 schools (4 overlap with 
core) 

- Site visits: Spring 2012 

- Teacher survey: Spring 2011 and 2012 


I 

I 


y 


14 


Case Studies of Schools Receiving School Improvement Grants: Final Report 


Sample Selection 

We selected a small, purposive sample of case study schools to support an in-depth examination of SIG 
schools in a variety of contexts. Although our sample is not designed to be nationally representative of 
all SIG schools due to the limited sample size, we did intend for the schools in our sample to vary on 
observable state, district, and school characteristics that might be associated with implementation 
patterns. At the school level, the following factors were taken into account in selecting the sample: 

• Tier I and Tier II schools. This study focused exclusively on Tier I and Tier II SIG schools because 
they were the only schools required to implement one of the four SIG intervention models (U.S. 
Department of Education, 2011). 

• SIG intervention model. Our sample did not include closure schools because these schools 
would not have been available for longitudinal study. Because the restart model was adopted 
for only 33 (4 percent) of the Tier I and II SIG schools from Cohort I (Hurlburt, Therriault, & Le 
Floch, 2012), we oversampled restart schools to ensure that at least a few of them were 
included in our sample. Otherwise, our sample was selected to approximately reflect the 
national distribution of SIG intervention models. 

• Grade level. At the recommendation of our Technical Working Group, we included only high 
schools and elementary schools in our sample to keep the dimensions of variation in the sample 
more manageable. These two levels merited inclusion for several reasons. Elementary schools 
were selected, in part, because of the large number of low-performing elementary schools. For 
example, 55 percent of SIG-eligible schools in the first cohort were elementary schools, whereas 
20 percent were middle schools (Hurlburt et al., 2011). High schools were selected, in part, 
because of the challenges faced by students and staff. For example, high schools are frequently 
organized into academic departments, serve students with a diverse set of postsecondary goals, 
and are populated by adolescents who often face adult responsibilities (Carnoy, Elmore, & 

Siskin, 2003; Flarvey & Housman, 2004; Le Floch et al., 2010). High schools also are widely 
perceived to be the most resistant to improvement strategies (and thus merit further inquiry) 
and are the focus of current policy interest, as a convergence of efforts on the part of private 
foundations, researchers, and advocacy groups has focused attention on high schools (Hess, 
2005; Hill, 2006; Yohalem et al., 2006). Middle schools face their own unique set of issues that 
deserve attention, particularly in relation to student behavior. However, given our limited 
sample size and the desire to be able to go sufficiently in-depth within each grade span, we felt 
that their exclusion from the study was an acceptable tradeoff. Our findings are thus not 
necessarily relevant to any low-performing middle schools and should not be used to make such 
generalizations. 

Additional school-level variables considered included school size, school urbanicity, demographics of 
enrolled students (i.e., percentage eligible for free or reduced-price lunch, percentage minority, and 
percentage ELL), and the size of the SIG award as a percentage of overall annual per-pupil spending. 14 


14 The percentage for each school was computed as the SIG school's annual SIG per-pupil award as a percentage of 
the per-pupil spending on instruction, support services (student support services, instructional staff, and school 
administration), and operation and maintenance for the year prior to the SIG award, for the district in which the 
school is located. This district measure is a proxy for per-pupil school-level spending (2009-10 base per-pupil 
spending figures from the Common Core of Data [http://nces.ed.gov/ccd/] are Consumer Price Index-adjusted to 
2011 dollars). 
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At the state level, we sought variation on the following variables: 

• Union policies (i.e., right-to-work versus unionized) 

• Level of base per-pupil spending on education (based on the year prior to SIG) 

• Level of SIG funding per school 

• Region 

Finally, we sought to select districts with multiple SIG schools— in part to facilitate analyses of the 
district role in the improvement process and in part for practical reasons (to limit data collection costs). 

Base sample. The base sample served as the pool from which we selected all other samples for the 
study. Using a two-step sampling approach, we selected 60 schools— 30 elementary and 30 high schools 
in 24 districts and 6 states— from the set of Cohort I SIG schools in 49 states and the District of 
Columbia. 15 We started by selecting 6 states from which the 60 base sample schools would be identified. 
In selecting these states, we sought variation in the key state-level dimensions. In addition, states were 
required to have (1) at least five SIG schools meeting the school-level criteria (i.e., Tier I or Tier II; 
elementary or high school; and implementing the turnaround, restart, or transformation model); and (2) 
at least one district with three or more SIG schools meeting the school-level criteria. We then selected 
the final base sample of SIG schools from the six selected states using an iterative process aimed at 
balancing urbanicity, SIG models, annual per-pupil funding, school size, and student demographics, 
while limiting (for cost reasons) the total number of districts represented in the sample. 

Core sample. From the 60 base sample schools, we selected 25 for the core sample, which was the focus 
of in-depth data collection in SIG Years 1 and 2. We sought to achieve a balance of intervention models, 
school levels, and nesting of case study schools within districts. We also sought to represent all six states 
and urbanicity categories from the base sample. Finally, we aimed to include schools with a range of SIG 
award sizes relative to overall annual per-pupil spending. The set of 25 schools included the following: 

• 13 elementary schools and 12 high schools in 13 districts 

• 16 urban schools, 5 urban fringe schools, and 4 rural schools 

• 13 transformation schools, 9 turnaround schools, and 3 restart schools 

Core subsample. From the 25 core sample schools, we selected 12 for the core subsample, which was 
the focus of data collection in SIG Year 3. The sample size was reduced to better target limited project 
resources on more in-depth analyses in a smaller set of schools. In choosing the 12 schools, we sought 
to maintain a balance of intervention models and school levels while also seeking variation in the 
following measures from our Year 1 data analyses that are hypothesized to be related to the 
improvement process (see Le Floch et al. [2014] for more discussion of these Year 1 measures): 

• Perceived external context. Analysis of Year 1 data revealed that core sample schools were 
situated in a range of community contexts, from "traumatic" environments characterized by 
reports of high crime, incarceration, abuse, and severe urban poverty to comparatively "benign" 
environments characterized by limited reports of crime, homes in good repair, and few reports 
of family instability. 

• Disruptions from the past. SIG was designed to spur dramatic and visible changes; however, 
Year 1 data suggested that core sample schools differed in the extent to which SIG coincided 
with a visible disruption of existing school norms and practices. Retrospective case studies 


15 At the time we selected the sample (March 2011), Flawaii had not released the list of their SIG schools. 
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suggest that low-performing schools that have improved their performance appreciably over a 
short time frame made dramatic changes from the status quo by signaling the urgent need for 
change, making visible improvements right away ("quick wins"), sharply focusing on instruction, 
and building a committed staff— often through releasing, replacing, or redeploying school 
personnel (Bryk et al., 2010; Herman et al., 2008). Interpreting these findings, turnaround 
advocates and researchers argue that to effectively address long-standing, intransigent patterns 
of low performance requires a dynamic, intensive, sustained change process that starts with a 
visible disruption of past practices (Hassel & Hassel, 2009). 

• Perceived role of SIG in the change process. Analysis of Year 1 data suggested that SIG's role in 
the change process was perceived by core sample schools to take one of two forms: as the 
primary impetus of change or as an effort that fit into an ongoing change process. For a third 
group of core sample schools, changes were described by respondents as "marginal" or 
"tweaks," indicating that, although the schools had made purchases with SIG funding, they had 
not launched a reform process. Case study research has documented how schools' contexts and 
experience with prior improvement efforts can influence how new initiatives are perceived and 
implemented (Scott, 2009). Based on this research literature, we hypothesized that the 
contribution of SIG is likely to be perceived differently in schools, depending on their context 
and history. 

• Perceived improvement in Year 1 of SIG implementation. Analysis of Year 1 data revealed that 
core sample schools varied in the extent to which respondents perceived that their schools were 
making progress after the first year of SIG implementation. The perceptions themselves, even if 
not necessarily supported by more objective measures of change (i.e., student test results, 
graduation rates, or other measures of student outcomes), could influence how the participants 
subsequently related to and participated in the improvement process. 

• Organizational capacity in Year 1 of SIG implementation. Analysis of Year 1 data suggested that 
core sample schools differed in their level of organizational capacity, as defined by evidence of 
the following eight intermediate outcomes hypothesized to be associated with higher levels of 
student achievement: teacher collaboration, safe and orderly climate, teachers' sense of trust, 
clear and shared goals, quality of leadership, use of data for instructional decisions, 
programmatic coherence, and locus of responsibility. In the context of a school improvement 
initiative, increases in these indicators may be viewed as signs of increased organizational 
capacity and thus as potential precursors to— or leading indicators of— subsequent 
improvements in student achievement. 

In considering these Year 1 measures, we sought to identify a subset of schools that would represent a 
range of different ways in which the change process may occur, as depicted in the conceptual 
framework and described in Chapter 1. The set of 12 schools included 5 elementary schools and 7 high 
schools in 8 districts and 5 states. 

Comparing Case Study Schools to SIG-Eligible and SIG-Funded Schools 
Nationwide 

Although our case study schools were not intended to be nationally representative, we sought to ensure 
that the core sample and core subsample shared some similarities with SIG-funded schools nationwide 
on observable characteristics. Compared to the population of SIG-funded Tier I and II schools, both the 
core sample and core subsample schools were more likely to be larger and high minority, with core 
sample schools also more likely to be urban (see Exhibit 2.2). The samples were more comparable to 
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SIG-funded schools on other variables. (Because of the similarities between SIG-eligible Tier I and II 
schools and SIG-funded Tier I and II schools, core sample and core subsample schools were similarly 
comparable to the overall population of SIG-eligible Tier I and II schools.) Some purposeful distinctions 
existed between the core sample and SIG-funded schools nationwide. Most notably, our sample 
included only elementary and high schools. With regard to SIG intervention models, the core sample and 
core subsample featured more turnaround and restart schools and fewer transformation schools as 
compared with SIG-funded Tier I and II schools. As previously discussed, closure schools were excluded 
from our samples. In addition, although SIG schools nationwide included alternative, special education, 
and vocational schools, the core and core subsample schools purposively included regular schools only. 


Exhibit 2.2. 

Characteristics of Cohort I SIG-Eligible Tier I and II Schools, 

SIG-Funded Tier I and II Schools, Core Sample Schools, and Core Subsample Schools 



SIG-Eligible Tier 1 
and Tier II 

Schools 
(yv = 2,141) 

SIG-Funded Tier 1 

and Tier II 

Schools 
(N = 826) 

Core Sample 
Schools 
(A/ =25) 

Core Subsample 
Schools 
(A/ =12) 

School Level (percentage of schools) 





Elementary 

21% 

24% 

52% 

42% 

Middle 

17% 

20% 

0% 

0% 

High 

51% 

49% 

48% 

58% 

Nonstandard 

11% 

7% 

0% 

0% 

School Type (percentage of schools) 





Regular 

86% 

91% 

100% 

100% 

Alternative 

11% 

7% 

0% 

0% 

Special education 

2% 

1% 

0% 

0% 

Vocational 

1% 

1% 

0% 

0% 

Urbanicity (percentage of schools) 





Large or middle-sized city 

54% 

59% 

68% 

58% 

Urban fringe and large town 

26% 

23% 

16% 

25% 

Small town and rural area 

20% 

18% 

16% 

17% 

Free and Reduced-Price Lunch (school 





average percentage of students) 3 

75% 

76% 

81% 

80% 

Race/Ethnicity (school average 
percentage of students) 3 

White 

19% 

17% 

8% 

11% 

African American 

43% 

46% 

42% 

40% 

Hispanic 

32% 

32% 

41% 

35% 

Native American 

2% 

2% 

4% 

8% 

Asian 

3% 

3% 

4% 

5% 

Total School Enrollment (school 





average) 

614 

676 

831 

859 
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Exhibit 2.2. 

Characteristics of Cohort I SiG-Eligible Tier I and II Schools, 

SIG-Funded Tier I and II Schools, Core Sample Schools, and Core Subsample Schools 

(continued from previous page) 



SIG-Eligible Tier 1 
and Tier II 

Schools 
(A/ = 2,141) 

SIG-Funded Tier 1 

and Tier II 

Schools 
(« = 826) 

Core Sample 
Schools 
(A/ =25) 

Core Subsample 
Schools 
(W =12) 

SIG Intervention Model (percentage of 
schools) 

Transformation 


74% 

52% 

58% 

Turnaround 


20% 

36% 

25% 

Restart 


4% 

12% 

17% 

School closure 


2% 

0% 

0% 


Source: 2009-10 Common Core of Data; state websites. 

Notes: Includes 2,141 Cohort I SIG-eligible Tier I and II schools and 826 Cohort I SIG-funded Tier I and Tier II schools in 49 states 
and the District of Columbia; 25 core sample schools in 6 states and 13 districts; and 12 core subsample schools in 5 states and 
8 districts. 

Percentage values for characteristics with multiple categories may not sum to 100 due to rounding. 

Nonstandard refers to those schools with a grade configuration not falling within the elementary, middle, or high school 
categories. 

a Student characteristics are weighted in proportion to the number of students enrolled in a school. 


Data Collection Activities 

Key data collection activities for the core sample and core subsample schools included the following: 

• Site visits. Site visits took place over 1.5 to 2.5 days in each of the 25 core sample schools during 
spring 2011 and 2012, with an additional visit in each of the 12 core subsample schools during 
spring 2013. We conducted interviews or focus groups with the principal, teachers, support 
providers, and other respondents, as well as interviews with officials of the district in which the 
school was located. Semistructured interview protocols guided these interviews and focus 
groups, and covered specific topics of interest consistently across respondents and years of data 
collection, allowing respondents to describe school improvement processes and policies in their 
own words. In addition to topics covered annually, site visits also focused on specific key areas 
each year, as discussed further below. 

o Supplemental telephone or in-person interviews with key staff. To supplement the 
spring site visits, we conducted shorter data collections, consisting of interviews with 
district representatives, principals, and instructional coaches, in each of the 25 core 
sample schools during fall 2011 and in each of the 12 core subsample schools during fall 
2012. 

• Teacher survey. We administered an approximately 10-minute Web-based teacher survey to all 
teachers in the 25 core sample schools in spring 2011, fall 2011, and spring 2012, and to all 
teachers in the 12 core subsample schools in fall 2012 and spring 2013. The survey covered 
selected topics for which we required data representative of all teachers in the school. 

• Interviews with state officials. Interviews conducted in spring 2011 and 2012 with state 
administrators responsible for overseeing the SIG program in the six sample states provided 
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insight on state-level decisions with regard to state funding, state contexts (e.g., legal 
constraints and flexibility), and state actions and technical assistance. 

In the remainder of this section, we provide additional details regarding the two primary data collection 
activities: site visits and teacher surveys. The complete set of data collection instruments for all activities 
can be found at http://www.air.org/project/study-school-turnaround. 

Site visits. Two researchers from the study team visited each school, with the lead site visitor 
responsible for conducting the interviews and the second site visitor responsible for taking notes. During 
some visits, the second site visitor conducted some of the interviews or focus groups, depending on the 
second visitor's experience. With the permission of interviewees, conversations also were audio- 
recorded. 

The study team aimed to interview the following respondents for each core sample and core subsample 
school, if applicable, during the spring site visits: the superintendent or district SIG director, the 
principal, one to two instructional coaches (e.g., mathematics and English), four teachers (two 
mathematics and two English teachers in the high schools), an external support provider (e.g., 
curriculum/instructional providers, school turnaround organizations, and charter management 
organizations), and a union representative. Site visitors also conducted focus groups with the following 
groups: the School Improvement Team, parents and community members, two groups of teachers (core 
and noncore subjects in high schools), and students (in high schools only). As previously noted, fall data 
collections were limited to interviews with a district representative, the principal, and one or two 
instructional coaches. To ensure a variety of perspectives on the schools' history and current change 
strategy, school personnel (generally, the principal or other school administrator) were asked to select 
teacher respondents with different levels of teaching experience, grade-level assignments, and subject 
areas taught (in the case of high schools). Respondent groups, such as parents/community members 
and high school students, were likewise selected by school personnel with instructions to include as 
diverse a group as was feasible. 

Exhibit 2.3 summarizes the number of respondents for each wave of site visits. For site visits to the 25 
core sample schools, the spring 2011 wave included 653 respondents; the fall 2011 wave, 106 
respondents; and the spring 2012 wave, 647 respondents. For site visits to the 12 core subsample 
schools, the fall 2012 wave included 44 respondents; and the spring 2013 wave, 267 respondents. Of the 
principals interviewed in each year, 44 percent were new to the school in 2010-11, 32 percent were 
new in 2011-12, and 17 percent were new in 2012-13. Among teachers interviewed in each year, 18 
percent were new in 2010-11, 2 percent were new in 2011-12, and 7 percent were new in 2012-13. 

Site visitors followed semistructured interview protocols that outlined key questions to ask of 
interviewees and provided follow-up probes. To build rapport with respondents, the interview structure 
also allowed for conversation and discussion. Site visitors remained flexible to follow up on themes that 
emerged during interviews that warranted more attention. Site visitors attempted to balance obtaining 
information from interviewees on the topics on which they were most knowledgeable while obtaining 
the perspectives of all respondents on key issues. 
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Exhibit 2.3. 

Number of Site Visit Respondents for Core Sample and Core Subsample Schools, 
2011-2013 



(n = 

Core Sample 

25 schools in 13 districts) 

Core Subsample 
(n = 12 schools in 8 districts) 

Respondent Group 

Spring 2011 

Fall 2011 

Spring 2012 

Fall 2013 

Spring 2013 

Superintendent 

3 

0 

2 

0 

0 

District SIG directors 

7 

8 

10 

3 

7 

Other district staff 3 

18 

9 

24 

0 

11 

Principals 

27 (12) d 

25 (8) 

25 (8) 

11 (2) e 

12 (2) 

Teachers 

323 (58) 

- 

271(5) 

- 

107 (7) 

Instructional coaches 

37 

37 

42 

15 

16 

Other school staff b 

22 

13 

91 

2 

32 

External support providers 3 

18 

14 

20 

7 

7 

Union representatives 

23 

- 

11 

- 

0 

Parents 

103 

- 

86 

- 

34 

Students 

72 

- 

65 

- 

41 

Total 

653 

106 

647 

44 

267 


Source: SST teacher survey, spring and fall 2011, spring and fall 2012, and spring 2013. 

Notes: Includes 25 core sample schools for spring 2011, fall 2011, and spring 2012; and 12 core subsample schools for fall 2012 
and spring 2013. The numbers of principals and teachers new to the school in the year in which they were interviewed are 
provided in parentheses. 

a Other district staff include assistant superintendents, school turnaround specialists, and curriculum/instructional support 
personnel. 

b Other school staff include school administrators, parent/community relations liaisons, and support personnel. 
c External support providers include curriculum/instructional providers, school turnaround organizations, and charter 
management organizations. 

d One school in the core sample is divided into four academies, each with its own principal. Three of these principals were 
interviewed, hence a total of 27 principals in the core sample for spring 2011. 
e The principal at one school in the core subsample was not interviewed in fall 2013. 


All interviews and focus groups adhered to the following topics, which were consistently addressed 
across all respondent groups and years of data collection: 

• Respondent's background 

• Respondent's role and responsibilities in the school 

• School context (demographics, strengths, challenges) 

• Key improvement activities in place or planned at the school 

• Perceptions of SIG and the change process 

In addition to these topics, each year of data collection also featured different areas of focus, informed 
by the stage of SIG implementation and findings from the preceding year of data collection. Site visits 
during Year 1 (spring 2011) focused on gathering contextual background information, including the 
school's reform history, the rationale behind SIG implementation in each of the 25 core sample schools, 
and initial improvement activities. Year 2 site visits (fall 2011 and spring 2012) focused on school 
leadership and organizational capacity-building activities, which may be particularly important for 
sustaining improvements beyond the short-term infusion of SIG funds. Year 3 site visits (fall 2012 and 
spring 2013) to the 12 core subsample schools captured respondents' reflections on the change process, 
both over the three years of SIG and during the most recent school year (2012-13). These data provide 
information on the trajectory and pace of the change process in these schools, as well as the potential 
sustainability of improvements after the end of SIG. 
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Teacher survey. To supplement the qualitative data, we administered surveys to all teachers in the 25 
core sample schools during the spring of the first data collection year (2010-11) and during the fall and 
spring of the second data collection year (2011-12); teachers in the 12 core subsample schools also 
responded to surveys in the fall and spring of the third data collection year (2012-13). 16 All spring 
surveys included items on the following areas: shared goals, shared values, instructional leadership, 
program coherence, teacher-teacher trust, principal-teacher trust, school resources, and student 
behavioral issues. Additional questions were included in the spring surveys for Years 2 and 3 to gather 
information about topics we wanted to explore further, including school improvement planning (added 
in Year 2), professional learning (included in Year 2 only), and perceptions of school change and 
perceptions on sustainability (added in Year 3). Items collected but not used in the Year 1 analyses, such 
as those related to school commitment and self-efficacy, were removed after the spring 2011 survey. 

The Year 2 fall survey focused on professional learning, and the Year 3 fall survey focused on school 
improvement planning and student engagement. When initially designing the teacher survey, we 
borrowed items developed for other studies, which demonstrated that they could be used to create 
reliable scales. Most scales were from teacher surveys developed by the Chicago Consortium on School 
Research (CCSR), but others were developed by staff from the American Institutes for Research for other 
national studies of school reform. 17 Subsequent items added to the teacher survey were created for the 
purposes of this study. 

Exhibit 2.4 presents the teacher response rates and distribution of response rates across schools for 
each survey administration. Because the teacher survey was designed to characterize broader teacher 
perceptions of activities and conditions in each school, schools with low response rates were excluded 
from survey analyses. Based on a series of exploratory analyses, we excluded survey results for schools 
with less than a 50 percent response rate (for more information, see Le Floch et al. [2014]). Year 1 
survey analyses included 21 core sample schools, and Year 2 survey analyses included 23 core sample 
schools. Year 3 survey analyses included all 12 core subsample schools. 18 


16 Survey windows ranged from 8 to 13 weeks during each administration. 

17 For additional information about survey items that were drawn from existing sources, see CCSR's survey 
documentation (https://ccsr.uchicago.edu/surveys/documentation) and the teacher surveys for the National 
Longitudinal Study of NCLB (www.air.org/topic/education/study-of-school-turnaround-teacher-survey-nls-nclb). 

18 The spring 2011 survey was administered electronically without any direct support from school personnel, which 
may have contributed to lower teacher response rates. From the fall 2011 survey onward, we enlisted the help of a 
survey coordinator in each school, who arranged group administrations and assisted with follow-up. This strategy 
helped boost response rates, although teachers in a subset of schools unfortunately continued to be non- 
responsive throughout the study. Teacher surveys were confidential, but incentives to teachers of any kind were 
prohibited. 
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Exhibit 2.4. 

Teacher Survey Response Rates, by Survey Administration 



Core Sample 

Core Subsample 


Spring 2011 

Fall 2011 

Spring 2012 

Fall 2012 

Spring 2013 

Number of Responding Teachers 

794 

750 

878 

455 

485 

Number of Surveyed Teachers 

1,280 

1,183 

1,218 

630 

591 

Overall Response Rate 

62% 

63% 

72% 

72% 

86% 

Response Rate Range (number of schools) 






75% to 100% 

5 

12 

16 

7 

10 

50% to 74% 

16 

9 

7 

5 

2 

25% to 49% 

3 

3 

2 

0 

0 

0% to 24% 

1 

1 

0 

0 

0 


Source: SST teacher survey, spring and fall 2011, spring and fall 2012, and spring 2013. 

Notes: Includes 25 core sample schools for spring 2011, fall 2011, and spring 2012; and 12 core subsample schools for fall 2012 
and spring 2013. 


Exhibit 2.5 presents the characteristics of the teacher survey respondents in our analysis samples, in 
addition to information on the intervention model, school level, and urbanicity of the teachers' schools. 


Exhibit 2.5. 

Characteristics of Teacher Survey Respondents 



Spring 2011 

Spring 2012 

Spring 2013 

School Level 

N 

Mean 

N 

Mean 

N 

Mean 

Teaches elementary school (percentage) 

216 

30.9% 

298 

33.9% 

135 

27.8% 

Teaches high school (percentage) 

482 

69.1% 

580 

66.1% 

350 

72.2% 

Intervention Model 







Teaches in a restart school (percentage) 

50 

7.2% 

90 

10.3% 

62 

12.8% 

Teaches in a transformation school (percentage) 

422 

60.5% 

598 

68.1% 

354 

73.0% 

Teaches in a turnaround school (percentage) 

226 

32.4% 

190 

21.6% 

69 

14.2% 

Urbanicity 







Teaches in a rural school (percentage) 

82 

11.7% 

114 

13.0% 

57 

11.8% 

Teaches in a nonrural school (percentage) 

616 

88.3% 

764 

87.0% 

428 

88.2% 

Teacher Background Characteristics 







Master's degree (percentage) 

341 

51.0% 

430 

52.4% 

247 

54.3% 

Years of experience 

693 

11.7 

863 

12.0 

476 

11.3 

Years of experience in current school 

682 

5.4 

863 

6.5 

456 

6.2 


Source: SST teacher survey, spring 2011, 2012, and 2013. 

Notes: Includes 21 core sample schools (11 elementary schools, 10 high schools) for spring 2011; 23 core sample schools (13 
elementary schools, 10 high schools) for spring 2012; and 12 core subsample schools (5 elementary schools; 7 high schools) for 
spring 2013. 

Percentage values for characteristics with multiple categories may not sum to 100 due to rounding. 
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Overview of Analytic Techniques 

Site Visit Analyses 

The qualitative analyses of site visit data followed a five-stage process: 

• Stage 1: Directly after site visits, site visitors entered descriptive information about the site visits 
(e.g., number of completed interviews, data collection challenges, a description of school 
context) into a preliminary Web-based data repository. 

• Stage 2: Based on the preliminary data capture and our conceptual framework, we developed, 
pilot-tested, and refined qualitative codes. 

• Stage 3: Analysts coded transcripts of the interviews and focus groups using Atlas.ti®. 19 

• Stage 4: Analysts compiled coded site visit and survey data in a second Web-based data 
repository, synthesizing findings for each school. 

• Stage 5: Analysts conducted cross-case analyses, based on the repository data. 

Stage 1: Preliminary data capture. Site visitors completed the preliminary data capture through a Web- 
based software program (SurveyGizmo in spring 2011 and SurveyMonkey® for all subsequent rounds of 
data collection). Using a Web-based platform allowed site visitors to access the data capture while they 
were still in the field, thus facilitating prompt entry of site visit data. The purpose of the preliminary data 
capture was to systematically record the details of the site visit while they were still vivid. This platform 
did not serve as a primary analytic tool but rather ensured that site visitors communicated key features 
of the site visit to other study team members, highlighted unanticipated issues, and noted gaps in data 
collection that would require follow-up. The preliminary data capture template asked site visitors to 
report case information pertaining to five topics: (1) site logistics, (2) SIG school characteristics, (3) site 
visit participants, (4) the school environment, and (5) first impressions of school improvement efforts. 

Site visitors were encouraged to complete all preliminary data capture activities while on-site but were 
required to finalize the preliminary data capture within two weeks of each site visit. After site visitors 
completed the preliminary data capture entry, members of the senior leadership team reviewed the 
entire entry. If reviewers identified inconsistencies or responses that seemed biased or incomplete, the 
reviewers required site visitors to revise them. 

Stage 2: Developing and piloting codes. Guided by our conceptual framework, we constructed a 
preliminary draft code list in spring 2011 based on: (1) key components of the conceptual framework, 

(2) regulatory requirements of SIG schools, and (3) topics that were mentioned by respondents and 
described in the preliminary data capture. 

Subsequent to determining the overall approach to coding and drafting the initial code list, we piloted 
the codes with a subset of qualitative data to determine whether the set of codes covered the topics 
reflected in the data, whether they were of an appropriate grain size, and whether the definitions in the 
codebook were clear. In the subsequent years of data collection, we revised the codebook, as needed, 


19 Across the three years of data collection, audio recordings were collected for all focus groups and all but five 
interviews, from which near-verbatim written transcripts were produced. For the five interviews that were not 
recorded, analysts relied on the site visitors' field notes. Prior to coding, lead site visitors reviewed the transcripts 
and field notes for their respective schools, and revised the records for clarity (i.e., explaining acronyms, 
identifying roles of individuals named in the interviews and focus groups), as needed. 


24 


Case Studies of Schools Receiving School Improvement Grants: Final Report 


to reflect emerging themes from the site visits, as well as issues unique to the ELL and rural-special topic 
samples. See Appendix A for the full list of codes and their definitions. 

Stage 3: Coding. To ensure that our data were coded consistently and reliably, the coding stage involved 
a multistep process that included training, weekly assessments of interrater agreement, frequent 
debriefing, and review of coded data by senior staff. In Year 1, all analysts who contributed to coding 
study data were required to participate in two training sessions: (1) a half-day webinar that focused on 
Atlas.ti®, the qualitative software program we used to code our data; and (2) a training session covering 
the definitions of all of the codes (see Appendix A). In subsequent years, experienced coders attended a 
half-day meeting to review key concepts of Atlas.ti® and revisions to the codebook, and new coders 
attended a separate two-hour training, in addition to the half-day training. 

Analysts then coded the transcripts for every interview and focus group. The unit of coding was a 
segment of text reflecting a given construct. In some cases, this consisted of one or two sentences, in 
other cases, one or two paragraphs. Analysts were trained to capture comparable segments of text for 
each coded passage, including enough adjacent text to enable a researcher to understand the data 
when a coded passage was retrieved from an interview or focus group. 

Assessing interrater agreement throughout the coding process entailed the following procedures: 

• Formal interrater agreement checks, conducted during the course of the coding process (two 
times in spring 2011 and once each in spring 2012 and 2013), required analysts to independently 
code passages from an interview transcript. These passages, selected by the two senior staff 
leading the task, were chosen to reflect a diverse set of codes, including those that were the 
most conceptually challenging. To pass an interrater agreement check, analysts had to match at 
least 75 percent of their codes to a key created by the two senior staff, while limiting any 
additional codes added beyond those in the key to no more than 25 percent of the total number 
of codes in the key. No analysts were permitted to code data until they passed the initial check. 
Analysts who did not meet this standard after the first attempt were required to repeat the 
process with a new passage of data until successful completion. Following each attempt, the 
task leaders reviewed the interrater agreement checks individually with the analysts to further 
build expertise and familiarity with the code list and coding strategies. Exhibit 2.6 presents the 
passage rates for each interrater agreement check. 

• In between formal interrater agreement checks, analysts conducted partner checks to review 
each other's work and maintained a message board to capture questions and concerns. In 
addition, weekly meetings focused on actual data pulled from interviews and focus groups that 
the analysts found difficult to code, which helped to highlight coding challenges and reach 
consensus on how to proceed. Decisions about necessary changes to code definitions, coding 
procedures, or the addition of new codes were documented by the coding task leaders and 
circulated among team members. The analysis team rarely added new codes during the coding 
process to avoid coders having to revisit previously coded data, thus creating workflow 
inefficiencies. When new codes were necessary, the coding task leaders reviewed all data files 
to ensure that analysts had applied the new codes appropriately and consistently across 
transcripts. 
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Exhibit 2.6. 

Interrater Agreement Check Passage, 2011-2013 


Spring 2011 


Spring 2012 


Spring 2013 


Check 1 

Passed on first attempt 
Passed on second attempt 
Passed on third attempt 
Passed on fourth attempt 
Total number of coders 


Check 2 

Passed on first attempt 
Passed on second attempt 
Passed on third attempt 
Total number of coders 


1 

4 

1 

6 a 



a Two coders were not involved in the entire coding period, leaving six active coders at the time of the second interrater 
agreement check. 


• Prior to their use in the next stage of analysis (Stage 4), all data were reviewed by the coding 
task leaders. As part of this ongoing review process, the task leaders examined all coded 
interview and focus group transcripts in a single case study school, checking coded passages for 
accuracy and consistency both across the individual case and against the coding guidelines. 
Where inconsistencies were identified, the task leaders communicated with the analyst to make 
the required revisions. 

After all of the interview and focus group data were coded, analysts used Atlas. ti® to run queries that 
helped sort the data (e.g., by code or group of codes). These queries served as the foundation for the 
data repository (Stage 4). 

Stage 4: Data repository. After all data from a case study school were coded, analysts entered analyst 
notes and coded passages in a Web-based, password-protected data repository in the SurveyMonkey® 
platform. The data repository consisted of open- and closed-ended questions to summarize the data for 
each case. The topics addressed in the data repository aligned with our conceptual framework and 
included questions related to the topic areas featured in each year of data collection (i.e., school 
context, key improvement strategies, and perceptions of SIG and the change process for all three years; 
contextual background information and the rationale behind SIG implementation in Year 1; school 
leadership and organizational capacity in Year 2; and respondent reflections on the change process over 
the three years and during the most recent year [2012-13] and on sustainability of improvements after 
the end of SIG in Year 3). 

The data repository accomplished four main purposes: (1) provided a summary of the data on selected 
topics across all respondents for each school; (2) ensured that data were of a comparable level of detail 
and quality across cases, so that analysts could more easily review and compare the synthesized data 
across schools; (3) allowed analysts to easily view and download data for cross-case analyses; and (4) 
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created an "audit trail" 20 because analysts were expected to document the sources for the findings 
synthesized in each repository question. 

When responding to questions in the data repository, each analyst first ran queries in Atlas.ti®, 
identifying coded text that aligned with each question. The purpose of an Atlas.ti® query was to retrieve 
all the relevant data for a given code from a given case. The coded data consisted of excerpts of text, 
which the analyst had identified as corresponding to a given code. To facilitate the queries in Atlas.ti®, 
the question stem in the data repository included a list of codes (in brackets) that should be used. For 
example, to respond to the question below, the analyst would retrieve data for a given school that had 
been coded as SIG_model selection. 

15. Please describe the rationale for selecting the intervention model for this school. If 'f Download 
there are divergent reports from different stakeholders, please account for those as well. Please 
be sure to include the perspective of the district administrator.[SIG_model selection] 


Before entering data in the repository, the analyst determined if there were data to address the 
question— that is, whether respondents were given the opportunity to respond to questions on this 
topic. For certain analyses, analysts determined that the qualitative data for some schools were 
insufficient to support analyses, and these schools were excluded from the analysis. 

Next, the analyst reviewed and summarized the coded data for the case study school in paragraph form 
in the data repository. The analyst was asked to include information on how many and which types of 
respondents provided data on the case study school and to note reasons why data may not have been 
reported. If there were divergent views within a given school (i.e., respondents who expressed 
conflicting views), the analyst documented these as well. The following excerpt (an actual data 
repository entry for one school) illustrates how analysts documented which respondents provided data 
on a given topic and how many respondents expressed consistent views: 

The school 21 principal and district administrator both affirmed that they chose the 
Turnaround model because they wanted to replace teachers, and the model gave them 
the leverage to remove 50% of the staff. The principal also claimed that in conversations 
with parents, "The thing that came up was that the teachers weren't good." He also 
noted that there had been frequent misassignments of teachers (e.g., a kindergarten 
teacher teaching seventh-grade English). In focus groups and interviews, teachers, 
instructional coaches, and other administrators reported that they had not been involved 
in the SIG application process and could not report on the rationale for the model 
selection. 

Closed-ended questions in the repository probed the dimensions of implementation outlined in our 
conceptual framework (coherence, divergence from prior practice, buy-in) and were presented as a 
rubric rating scale. When selecting a rating for the case study school, the analyst was required to 
document which respondents had provided data on a given topic and to justify the rating (see Exhibit 
2.7). 


20 An audit trail is documentation that creates a chain of evidence that may be inspected by other researchers who 
seek to reproduce the findings (see Lincoln & Guba, 1985). 

21 All descriptions of the core sample schools use pseudonyms, and identifying characteristics of schools and 
individuals have been masked (e.g., the reported gender of respondents was randomly assigned). 
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Exhibit 2.7. 

Sample of the Online Data Repository 


30. If respondents comment on the safety and orderliness of the school during Spring 2013 interviews or focus groups, 
please note this shortly after the interview or focus group. A few words for each respondent in list form should suffice 
(e.g., principal called school "very safe," with "no reports of crime in the surrounding neighborhood") 

All respondents at Gale Secondary suggested that the school was safe and orderly, with most 
respondents (instructional coach; three interviewed teachers; four focus group teachers) answering 
with a simple, emphatic, "yes". One interviewed teacher highlighted the "family-like atmosphere" of the 
school, while the principal described how issues external to the school did not permeate through the 
walls of the school: "We maintain a pretty good — we know who those folks that are involved out on the F 
streets that are here. And we keep pretty close tab on that and I've tried to keep a good relationship 
built with them to try and minimize anything that would happen here.” One teacher in a focus group 
acknowledged while there are some community-related issues that are worrisome, "we haven't found 
any sort of fear here at the school." A teacher in a separate focus group said: "Extremely. Yes, you 
know, especially with things going on in the news, we feel extremely safe. We've had drills that deal 

ivith cafah/’ owoni taaohor L-nr»\*/o tr> va/q hawa /»ricic nlanc in r\lapo etaff HawoUnmont \a iq'wq 

31. Based on the data noted above, how would you rate the safety and orderliness of the school in Year 3? 

Safe/Orderly. At least two respondents explicitly described the school as safe or as having none or few behavior problems among students, and no 
respondents disagreed or made statements to the contrary. 

O Mixed. Respondents made contradicting or differing statements about the school’s safety and about student behavior. 

Unsafe/Disorderly. At least two respondents described a feeling of being unsafe or behavior problems among students, and no respondents disagreed 
or made statements to the contrary. 

O Insufficient evidence 


After the analyst completed the data repository for a school, one of the site visitors read the entire 
entry, ensuring that all responses were detailed, documented data sources, provided justification for 
summary statements, and were an accurate depiction of the school. In addition, senior researchers 
reviewed responses across schools to ensure that the level of detail across schools was comparable. 

Stage 5. Cross-case analyses. When the data repository was complete, analysts reviewed the data 
across all schools for a given topic (e.g., initial SIG processes, teacher replacement, or leadership). For 
most analyses, analysts developed classification schemes to categorize schools on the basis of the data 
and the research base associated with each topic. For example, with regard to the perceived 
performance problems in each school, analysts established decision rules to distinguish between schools 
in which respondents assumed internal responsibility for the history of low performance and schools in 
which respondents focused on challenges external to the school. Analysts extracted the relevant data 
from the data repository, reviewed the evidence for each school, and classified the school accordingly. 
Analyses that are related to perceived changes from years prior to SIG implementation or over the three 
years of the SIG program exclude the responses of principals and teachers who were new to their 
schools. 

After an analyst classified the schools, a second analyst reviewed the coded data and also classified the 
school, providing a measure of interrater agreement. Across the school-level classifications, the first set 
of interrater agreement ratings ranged from 72 to 100 percent. In cases of disagreement, a senior 
researcher reviewed the ratings and resolved discrepancies. When classifications were complete, the 
site lead for each school was required to review the data and the rating. When there was a discrepancy 
between the analyst's rating and that of the site visit lead, study team members jointly reviewed the 
data, returning to the original coded data if necessary. Thus, for every school-level classification included 
in this report, multiple researchers reviewed and approved the analysis. For further details on each set 
of decision rules and definitions of school-level classifications, see Appendix B. 
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After the initial school-level classifications were identified, analysts examined associations between 
selected classifications, based on our conceptual framework. For example, were schools in which 
respondents reported substantial progress located in "benign" environments characterized by limited 
reports of crime, homes in good repair, and few reports of family instability? To conduct these analyses, 
analysts created two-way tables to determine if there were relationships between sets of school-level 
classifications. 

In summary, all qualitative analyses were guided by principles of high-quality qualitative analyses: (1) 
clear standards of evidence for codes and ratings, (2) documentation of an "audit trail," (3) procedures 
for verifying consistency of data across cases, and (4) measurement of interrater agreement in coding. 

Use of Quotations. Throughout this report, we incorporate direct quotations from respondents, which is 
a standard technique in qualitative case study research (Miles, Fluberman, & Saldana, 2013). There are 
two primary reasons for their use, one methodological and one stylistic. 

With regard to methodology, we include direct quotations from respondents to lend more transparency 
to our constructed measures and allow the reader to better judge whether the measures appear well- 
grounded in the data. As described above, we developed school-level categorizations that aggregate 
qualitative responses from multiple interview and focus group respondents (see Appendix B for a 
detailed description of each measure). To do so, we reviewed coded data and categorized schools based 
on documented decision rules, thereby condensing the qualitative data into a more systematic and 
quantitative measure. This conversion process sometimes involved making judgments about whether 
specific quotations from study respondents met the established thresholds for a particular 
categorization. By providing example quotations with explanations of how we categorized schools rather 
than merely describing these categorizations in the abstract, we more concretely illustrate how we 
applied the decision rules. These quotations were not selected randomly but rather to be illustrative of 
the types of quotations associated with particular decision rules so that the reader can more fully 
understand each categorization. 

With regard to style, quotations enhance the transparency, clarity, and relevance of this study, which is 
based largely on qualitative data. These data uniquely provide detailed, contextual information that can 
convey meaning through illustrative examples. Quotes were purposefully selected to enrich the findings 
arrived at through systematic, carefully-documented analyses. As with the quotations selected for 
methodological reasons, quotations selected for stylistic reasons are not representative of all quotations 
in our data. It is important to bear in mind that these quotations are not used to validate an analysis, or 
to "prove" a particular finding, nor should they be construed to represent the sole evidence on which a 
finding was based. They are only meant to enrich a particular finding by conveying richer contextual 
information that is, by necessity, masked from the study's more systematic aggregate measures. 

Teacher Survey Analyses 

The surveys were designed to measure constructs pertaining to the school as a whole related to 
contextual influences, selected improvement actions, and dimensions of implementation (see Exhibit 
1.2 in Chapter 1). We assessed the quality of the teacher survey scales by conducting a confirmatory 
factor analysis on the items separately for each scale and by computing the scale reliability (Cronbach's 
alpha). Exhibit C-l in Appendix C shows the reliability and contributing items for each scale. 22 Scale 
scores were computed on the basis of the mean of the individual items composing each scale. Where 


22 Scale reliabilities were calculated using data from spring 2011 surveys administered to teachers from 21 of the 
25 core sample schools that exceeded a 50 percent response rate. 
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one or more items were missing, the scale was computed as the mean of the remaining items. 23 The 
school resources scale and the student behavior scale ranged from 1 (major challenge) to 4 (not a 
challenge). The other six scales ranged from 1 (strongly disagree) to 4 (strongly agree). For analyses of 
teacher collaboration and coherence of programs within the study schools, we relied on individual 
survey items, as the items for the respective analyses did not factor together to form reliable scales. 24 
Response options for the three stand-alone collaboration items ranged from 1 (never) to 4 (often), and 
the three stand-alone coherence items from 1 (strongly disagree) to 4 (strongly agree). See Appendix C 
for additional details about the teacher survey analyses. 


23 For example, if a teacher answered the first two items listed in the school commitment scale but did not answer 
the third item, his or her school commitment scale value would be the mean value of the first two items only. If 
another teacher skipped the first two items but answered the third, his or her school commitment scale value 
would be the value of the third item. Across scales, between two percent and four percent of teachers had a 
missing value on one or more scale items. 

24 The reliabilities of the scale formed by combining the three items related to teacher collaboration in Years 1, 2, 
and 3 were 0.66, 0.72, and 0.68. The reliabilities of the scale formed by combining the three items related to 
programmatic coherence were 0.57, 0.40, and 0.39. 
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Part II: Building Human Capital in SIG Schools 


At the heart of a school's capacity to improve outcomes for students are the knowledge, skills, and 
dispositions of the staff responsible for teaching those students and leading the improvement 
processes— that is, the individual and collective human capital of the adults in the school. The link 
between adult human capital and student outcomes has been documented in research and supported 
by theory (Aaronson, Barrow, & Sanders, 2007; Cohen & Ball, 1999; Hanushek, 1986), as has the limited 
human capital generally available to persistently low-performing schools (Beaver & Weinbaum, 2012; 
O'Day, Goertz, & Floden, 1995). In addition, many observers have noted that an emphasis on capacity 
building— and particularly on building the capacity of school personnel— may be particularly important 
for sustaining improvements beyond the short-term infusion of funds in grant programs such as SIG. In 
recognition of the need to enhance human capital in these schools, SIG policy requires staffing changes, 
professional learning activities, and partnerships with districts and external providers that are all 
ostensibly aimed at improving staff knowledge and skills. 

Part II of this report thus focuses on the strategies of the 25 core sample schools to build human capital 
as an avenue toward improving student success. For both leaders and teachers, we considered two main 
leverage points or strategic approaches: (1) changing the mix of individuals performing key functions in 
the school through staff replacement, role assignment, and addition of new positions; and (2) increasing 
the knowledge and skills of existing staff through professional learning. We incorporated data from the 
year prior to SIG (2009-10) and the first and second years of the SIG program (2010-11 and 2011-12) in 
these analyses. The primary data sources included the teacher survey administered in 2012, as well as 
interviews and focus groups conducted during site visits to all core sample schools. 25 

Part II consists of five chapters. Chapter 3 focuses on efforts to enhance school leadership by replacing 
the principal and expanding distributed leadership opportunities. Chapter 4 describes schools' efforts to 
build human capital by altering the composition of the teaching staff and others who provide support to 
teachers. These efforts included teacher replacement (both SIG mandated and otherwise) and the 
addition of new staff positions. The chapter also considers the challenges schools faced in recruiting and 
retaining high-quality teachers and identifies the supports their district provided to address those 
challenges. Chapter 5 looks at schools' efforts to build the capacity of their teachers through 
professional learning, including the range of hours spent in professional learning activities, the types of 
activities offered to teachers (job-embedded versus traditional), the structures that schools and districts 
established to support teacher learning (specifically around collaboration and data use), and the extent 
to which teachers perceived professional learning opportunities to be aligned with school needs and 
effective at changing their own practices. Chapter 6 explores how districts and external support 
providers contributed to SIG schools' efforts to build human capital. Flere, we describe in greater detail 
the types of support that districts and external support providers extended to schools, as well as the 
extent to which those supports aligned with the capacity-building approaches described above. We also 
discuss the extent to which school-level stakeholders perceived district and external provider supports 
to be positive for the school's improvement process. Chapter 7 concludes Part II with a synthesis of the 
various efforts that schools reported making in the areas above and consideration of whether some 
schools put forth more or less overall effort in their human-capital-building activities. 


25 Specific chapters of Part II detail which data are being included in the analysis of efforts to build human capital. 
More detailed descriptions are also available in Appendix B. 
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In Part II of this report, we find that the majority of case study schools were trying to build human 
capital using both staffing and professional learning strategies. Although it is clear that these SIG schools 
received numerous types of supports from their districts and a wide range of external support providers, 
it is less clear whether these supports will ultimately build human capital. In Part III of this report, we 
analyze the relationship between efforts to build human capital and changes in the organizational 
capacity of the school, as measured in Year 1 and Year 3 of the study. 
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Chapter 3: Building Human Capital Through 
Leadership Strategies 


In this chapter, we describe efforts to build human capital in the 25 core sample schools by (1) replacing 
the principal and (2) distributing leadership responsibilities among a greater portion of the existing staff. 

Case study research suggests that the principal plays a central role in leading major change efforts in a 
school (Edmonds, 1979; Hassel & Hassel, 2007; Herman et al., 2008; Picucci et al ., 2002; Rhim et al., 
2007; Johnson & Asera, 1999; Whiteside, 2006). As with leaders of any complex organization, principals 
have multifaceted roles that demand significant knowledge and skills. For example, principals make 
budgetary decisions and provide administrative oversight of instructional approaches and professional 
learning strategies. Principals also commonly play an important role in assessing school needs, setting 
goals, and implementing improvement strategies. SIG requires schools implementing the turnaround or 
transformation models to replace the principal if the principal has been at the school for more than two 
years prior to the first year of receiving SIG (U.S. Department of Education, 2010). 26 This requirement 
reflects the assumption that entrenched, ineffective leadership contributes to persistently low school 
performance and bringing in a new principal will help create the conditions necessary to improve. 

Although principals play a central role in leading major change efforts in a school, both empirical studies 
and theoretical explorations of leadership have also suggested that leadership in a school setting may 
come from multiple sources and be distributed across multiple individuals and structures (Elmore, 2000; 
Spillane, Halverson, & Diamond, 2004). Recent research in more than 100 U.S. schools showed that 
responsibility for school leadership functions was typically distributed among three to seven people, 
including administrators and specialists (Camburn, Rowan, & Taylor, 2003). This distributed leadership 
strategy reflects the assumption that leveraging existing knowledge and skills will enhance the collective 
leadership capacity in the school, empower staff to take ownership of the improvement process, and 
provide some continuity if there is leadership instability at the principal level. 


Box 3.1. Key Chapter 3 Findings 

• Most core sample schools (21 of 25) replaced their principal in 2009-10 or 2010-11. Between 
2009-10 and 2011-12, 9 of 25 core sample schools replaced their principal twice. 

• Half of the principals (10 of 20) who were new to their schools in 2010-11 or 2011-12 were 
described as an improvement over their predecessors by teachers, instructional coaches, 
school leadership teams, or parents. 

• In Year 2 of SIG, respondents in 7 of the 25 core sample schools reported having structures 
and opportunities for distributed leadership, respondents in 12 schools reported a moderate 
level of distributed leadership, and respondents at the remaining 6 schools provided little 
evidence of distributed leadership. 

• In 15 schools, respondents reported that administrators made an effort to establish or 
enhance structures to support distributed leadership during the first two years of SIG. 


26 Although schools implementing the restart model are not formally required to replace the principal, they 
typically do as part of the transition to a new governance structure (e.g., education management organization or 
charter management organization). 
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Principal Replacement 

Replacing a weaker principal with a stronger one may build organizational capacity if the new leader can 
create a shared direction, help build the knowledge and skills of the staff, redesign the organization to 
support the staff's work, and guide the "technical core" activities to manage curriculum and improve 
instruction (Leithwood, Harris, & Strauss, 2010). Although it is beyond the scope of this study to 
determine whether new principals were able to accomplish these "high-capacity" tasks, this section 
discusses the extent to which principals were replaced in the 25 core sample schools between 2009-10 
and 2011-12, and whether other stakeholders perceived these replacements as improvements over 
their predecessors. 

Most core sample schools (21 of 25) replaced their principal either in the year before SIG (2009-10) or 
in Year 1 of SIG (2010-11). New principals arrived at 11 of the 25 core sample schools in the year prior 
to SIG, and at 11 of the 25 core sample schools in the first year of SIG. 27 In Year 2 of SIG (2011-12), 9 
core sample schools replaced their principal. Even though schools implementing the restart model were 
not formally required to replace their principal, all three of our restart schools replaced their principal in 
Year 1 of SIG. All of the remaining core sample schools, including the ones that did not replace their 
principal, were implementing either the turnaround or transformation models, which required schools 
to replace their principal unless the principal had served at the school for less than two years. Box 3.2 
profiles a core sample school that acquired a new principal during the year prior to SIG (2009-10) and 
reportedly underwent positive changes under the new principal's leadership. 


Box 3.2. Rossignol Elementary: An Experienced Principal With a Clear Vision 
for Improvement 

Rossignol Elementary is a high-poverty, high-minority school located in a culturally diverse U.S. city. 
According to interviewed district and school stakeholders, in the years leading up to SIG, Rossignol 
suffered from weak and inconsistent leadership. A spate of inexperienced principals reportedly cycled 
in and out of the building, most of them lasting only a single year or less. Veteran school staff 
mentioned that one of the school's recent principals had been a particularly poor fit for the school 
and had sown a culture of distrust within the school community. Additionally, teachers described the 
school as "forgotten" and "neglected" by district leaders, who tended to assign the school teachers 
that other schools in the district did not want. 

Rossignol's leadership situation changed, however, during the year prior to SIG when a new leader 
took control of the school in what district and school staff characterized as an important turning 
point. Unlike most of her predecessors, the new principal had more than a decade of experience 
leading schools. She also had experience working in the district central office, a role that had 
familiarized her with the inner workings of the district bureaucracy and allowed her forge connections 
with key community partners. The principal reportedly leveraged this background to acquire new 
resources and supports for the school. For example, some of the principal's early priorities involved 
securing new instructional materials (e.g., science kits, technology) and engaging community partners 
to provide family outreach and support. One teacher commented, "If we didn't have [the new 
principal], we wouldn't have what we have. She has so many contacts. She knows what she's doing. 
Had they put another inexperienced principal here, we would be at the bottom." Another teacher, 


27 One school replaced its principal in 2009-10 and then again in 2010-11. 
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(continued from previous page) 

who expressed similar sentiments, underscored how the new principal had succeeded in acquiring 
resources that teachers had long been "begging for." 

Rossignol stakeholders universally characterized the new principal as a strong, influential, "hands-on" 
leader with a clear vision for improving the school. Multiple respondents credited her with building a 
cohesive teaching staff that was committed to realizing her vision, in part by helping individuals who 
were not on board with the vision find a position elsewhere. In spring 2012 (the end of SIG Year 2), 
interviewed teachers indicated that the principal had thus far employed a fairly directive leadership 
style to achieve her vision, and they spoke of her assertiveness in largely positive terms. For example, 
one teacher leader explained, "[At] the first leadership meeting, [the principal] said, 'If I'm seeking 
advice, I'll tell you. If I'm not seeking advice, I'm the boss.' I thought that was great. Now, we know 
what the deal is. You can often have a principal that seems like they want your input, but they don't. 
With [this principal], you know what you're getting." Several teachers attributed the principal's 
directive style to the school's dire situation. "I think it's that we need to get a whole lot of things in 
place, and that's the most efficient way to do it," one teacher surmised. Flowever, these teachers 
predicted that the principal might adjust her leadership style once the school had more systems and 
supports in place. Overall, school stakeholders looked ahead to their third and final year of SIG— and 
their fourth year with the same principal— with optimism for continued progress. 


During the three-year period between 2009-10 and 2011-12, 9 of 25 core sample schools replaced 
their principal twice. Three schools maintained the same principal for all three years, and the remaining 
13 schools replaced their principal once during that time period. Although a previous case study of SIG 
implementation suggested that districts struggled to identify school leaders with sufficient qualifications 
(McMurrer, 2012), most districts in our core sample appeared able to bring in principals whom they 
thought could be successful at turning around their schools. For example, the new principals in our 
sample were often experienced principals whom district administrators reported identifying and 
recruiting for their positions. Of the 20 principals who were hired in Year 1 or Year 2 of SIG, 14 were 
experienced leaders, most of whom had 5 to 10 years of experience as principals at other schools, in 
addition to longer term experience as teachers and/or assistant principals in low-performing schools. Six 
of the new principals had never held a principalship before, although two of them reported having 
previously been assistant principals at the same school. 

Most of the principals who were new to their schools in Year 1 or 2 of SIG reported that somebody from 
within the district had identified them for the position and had encouraged them to apply for it. For 
example, the principal at one school, a former district administrator, accepted the position after serving 
as assistant principal at another low-performing school in the district. Colleagues from the district 
recommended her because of her background (she was Latina and also had worked with that population 
in the past) and knowledge of the district's key initiatives. Similarly, at another core sample school, a 
district administrator and school staff member mentioned that the new principal had notably improved 
student performance at another elementary school over a span of three years. 

Half of the principals (10 of 20) who were new to their schools in 2010-11 or 2011-12 were described 
as an improvement over their predecessors by teachers, instructional coaches, school leadership 
teams, or parents (see Box 3.3). For example, multiple teachers at one school described the new 
principal as "reflective" and having "an open door policy," whereas they explained that the former 
principal would "just bark at us" and that student learning "didn't matter to him." In contrast, teachers 
in a focus group at another school reported that the new principal did not treat them with respect, and 
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an instructional coach described much less interaction between the principal and staff, noting that "a 
different administrator has a different way of doing things." 


Box 3.3. Perceived Change in Quality of Principal Leadership 

School classifications on perceptions of the change in quality of principal leadership for the 20 core 
sample schools that replaced their principal in Year 1 or Year 2 of SIG are based on Year 1 and Year 2 
interview and focus group data (see Exhibit B.l for more detail on the analytic procedures). For this 
analysis, respondents refer to individuals from the following respondent groups (excluding newly 
hired staff): teachers, instructional coaches, and school improvement team members. 

Quality of principal leadership improved 

• Respondents in at least two respondent groups described the change in principal in positive 
terms; AND 

• No more than one respondent described the change in principal in negative terms. 

Quality of principal leadership did not change 

• Respondents did not comment on the quality of the change in principal leadership or 
described the change in neutral terms. 

Quality of principal leadership declined 

• Respondents in at least two respondent groups described the change in principal in negative 
terms; AND 

• No more than one respondent described the change in principal in positive terms. 


Survey data from Year 1 and Year 2 provide additional evidence of how teachers perceived how the 
quality of leadership changed once the principal was replaced. We used two survey scales— principal- 
teacher trust and principal instructional leadership— to compare teachers' perceptions of their principals 
in Years 1 and 2 of SIG for the nine core sample schools that replaced their principals in Year 2 of SIG 
(see Exhibit 3.1). 28 Both survey scales included questions measured using a 4-point Likert scale. In four 
of the five schools where interview and focus group respondents described their Year 2 principal as an 
improvement over their predecessor, the Year 2 principal-teacher trust and principal instructional 
leadership scales were also higher relative to Year 1. In the fifth school, the principal-teacher trust scale 
was higher but the principal instructional leadership scale was lower. In the two schools where the new 
principal was not described in favorable terms by site visit respondents, the two survey scales were 
lower in Year 2 relative to Year 1. 


28 This supplementary analysis reflects changes in survey scales related to principal leadership from Year 1 to Year 
2, only for the nine schools that had new leaders in Year 2. Because we have no pre-SIG survey data, we cannot 
present analogous changes in survey scales for the principals who were new to their schools in Year 1 of SIG. This 
analysis is intended to provide additional information on the perceived quality of new principals, but it is distinct 
from the qualitative analysis described in Box 3.3. 
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Exhibit 3.1. 

Perceived Improvement of New 2011-12 Principals Compared With 2010-11 Principals 



Qualitative Perceptions 
of Principal Improvement 
Relative to Year 1 

Principal-Teacher Trust 

Principal Instructional 
Leadership 

Year 1 

Year 2 

Year 1 

Year 2 

Gale Secondary 

Improved 

2.9 

3.4 

3.1 

3.4 

Tyron Elementary 

Improved 

2.4 

2.6 

2.6 

2.9 

Island Bay Elementary 

Improved 

1.2 

3.0 

1.5 

2.9 

Meribel High 

Improved 

3.0 

3.1 

3.4 

3.1 

Aerovista High 

Improved 

2.8 

3.0 

2.7 

3.0 

West Marble High 

Neutral 

2.7 

3.0 

2.7 

2.6 

Sterling Slope Elementary 

Neutral 

3.6 

3.2 

3.7 

3.5 

Coral High 

Declined 

3.3 

2.6 

3.3 

2.9 

Melon Elementary 

Declined 

3.4 

2.7 

3.2 

3.2 


Source: SST respondent interviews and focus groups, spring 2012; SST teacher survey, spring 2011 and 2012. 

Notes: Includes the 9 core sample schools that had a new principal for the 2011-12 school year. All school names are 
pseudonyms. 

Rows are shaded to indicate schools in which the Year 1 principal had been reportedly replaced in a deliberate effort to 
improve capacity. In other schools, principals were reportedly replaced because of within-district transfers or retirement. 


Four of the five principals whom site visit respondents perceived to be an improvement over their 
predecessors reportedly had been brought in to the school in a deliberate effort to improve capacity 
(indicated by the shaded rows in Exhibit 3.1). 29 In three of these cases, district administrators specifically 
explained that the leadership change occurred because the prior principal was a "bad fit" or unqualified 
for the job. For example, at one school, the district administrator explained that the principal who was 
hired in Year 1 of SIG was not up to the challenges of school turnaround: "We've lent a very heavy level 
of support, particularly with that school leader this year. But, what we didn't see was a certain level of 
leadership to have managed the. ..many moving parts to manage this school year." Five other core 
sample schools also had new principals in Year 2, but their predecessors reportedly left for personal 
reasons or were promoted to other positions in the district, rather than having been removed to try to 
improve leadership at the school. 

Despite the fact that our qualitative and survey data indicate that in some of our core sample schools 
the replacement of the principal was perceived positively, principal replacement as an improvement 
strategy is not without risk. For example, as noted above in at least a few schools, it did not appear that 
principal replacement decisions were deliberately made with an eye toward improving leadership, and 
in at least a few schools, it appeared that the replacement principal was not a good fit for the school. 
Furthermore, principal replacements sometimes occurred in contexts in which leadership instability was 
already identified as part of the performance problem. Over the three-year period between 2009-10 to 
2011-12, only 3 of the 25 core sample schools maintained the same principal, while 9 had two 
leadership changes over three years and the remaining 13 had one change. Analysis of site visit data 
from Year 1 of SIG revealed that respondents from 10 of the 25 core sample schools reported that high 
turnover in leadership had been a challenge prior to SIG and attributed their history of low performance, 
in part, to leadership instability. As a teacher in a focus group from one such school explained, "I think 


29 Schools were classified as having replaced their principal as a deliberate effort to improve capacity if reported by 
a district administrator or at least one school-level respondent (principal, teacher, or instructional coach) (see 
Exhibit B.2 for more detail on the analytic procedures). 
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one of the things that has been plaguing this school since the beginning is inconsistency. You get 
administrators and teachers constantly changing, and it's really hard... .Every two years we get a new 
administrator." A colleague in the same focus group concurred: "It is frustrating to have these constant 
changes for teachers and administrators, when we have some things that we'd maybe like to talk about 
at the end of the year and discuss for next year.. .and we don't even know if our principal is coming 
back." 

Distributed Leadership 

Distributed leadership— also known as shared leadership— is a leadership model in which principals 
have multiple school staff members take on leadership roles at the school. Scholars of distributed 
leadership advance numerous rationales for such an approach. First, school leadership (particularly 
among the lowest-performing schools) is too complex a job to be accomplished by a single, "heroic" 
leader (Camburn, Rowan, & Taylor, 2003). Distributing responsibility across a wider range of individuals 
can more effectively leverage staff expertise, improve collective decisions, and promote implementation 
of those decisions. Timperley (2005) explains that distributed leadership "has the potential to build 
capacity within a school through the development of the intellectual and professional capacity of the 
teachers" (p. 3). In low-performing schools, such as those in our sample, principal turnover is common. If 
principals are able to distribute leadership opportunities across a wider set of teachers and other 
support staff (e.g., assistant principals, instructional coaches), then these efforts could add to the 
collective organizational capacity of the school by building the knowledge and skills of individual school 
staff and by better preparing the school for changes in principal leadership. 

School leaders' approaches to sharing leadership responsibilities among school stakeholders and 
engaging school staff in decision-making processes can vary. Efforts to distribute leadership can include 
forming leadership teams and committees, creating leadership roles for teachers, making decisions 
jointly with other staff, and delegating decision-making authority to others. Studies of distributed 
leadership have focused on formal and informal leadership structures and roles (which Spillane [2005] 
refers to as the "leader-plus" view), as well as the nature of the interactions among leaders, followers, 
and their situations (FHallinger & Fleck, 2010; Spillane, 2005). It was beyond the scope of this study to 
conduct a systematic inquiry of the nature of interactions among leaders and followers. Our analyses 
focus primarily on the "configuration" of the distributed leadership activities in each of the 25 core 
sample schools, similar to the work of Camburn et al. (2003). Using qualitative data from Year 2 of SIG, 
we examined the extent to which school respondents perceived that their schools had six hypothesized 
structures and opportunities for distributed leadership (see Box 3.4), including: 

• Clear set of responsibilities for school leadership team 

• School leadership team membership open to nonadministrative school staff 

• School leadership team involvement in decision-making process 

• Nonteaching staff (e.g., assistant principals, instructional coaches) involvement in decision- 
making process 

• Teacher involvement in decision-making process 

• Opportunities for teachers to assume leadership roles 
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The features that we identified are consistent with measures employed in other studies (Camburn et al., 
2003; Louis et al., 2010). 30 Our index of distributed leadership is a summative measure; we hypothesized 
that simply having a leadership team was not sufficient to promote distributed leadership schoolwide. 
Rather, this team needed to have a decision-making role and ideally would include representation from 
staff across the school, not only teachers. In addition, we sought to reflect informal leadership positions. 


Box 3.4. Perceptions of Distributed Leadership in Core Sample Schools 

School classifications on perceptions of distributed leadership are based on Year 2 interview and focus 
group data (see Exhibit B.3 for more detail on the analytic procedures). Because there were no 
natural breaks in the distribution, cut points were set to divide the schools roughly into thirds. For this 
analysis, respondents refer to individuals from the following respondent groups: principals, teachers, 
instructional coaches, and school leadership team members, although the identification criteria for 
individual features associated with distributed leadership— specifically the number and type of 
respondents— varied by feature. Clear set of responsibilities for school leadership team and school 
leadership team open to nonadministrative school staff are based on at least one member of the 
school leadership team; school leadership team involvement in the decision-making process are based 
on the principal; nonteaching staff involvement in the decision-making process are based on at least 
two from the following respondent groups— instructional coach or other school administrator (e.g., 
assistant principal); and teacher involvement in decision-making process and opportunities for 
teachers to assume leadership roles are based on at least two teachers. 

Structures and opportunities to support distributed leadership 

• Respondents perceived that the school had at least five of the six features associated with 
distributed leadership. 

Moderate structures and opportunities to support distributed leadership 

• Respondents perceived that the school had three or four of the six features associated with 
distributed leadership. 

Little/no structures or opportunities to support distributed leadership 

• Respondents perceived that the school had one or two of the six features associated with 
distributed leadership. 


Respondents at 7 of the 25 core sample schools reported that structures and opportunities for 
distributed leadership existed during Year 2 of SIG. Exhibit 3.2 lists the perceived features of distributed 
leadership for each core sample school. 

• Leadership team membership is open and varied. Leadership team members in these seven 
schools reported that membership was open to nonadministrative staff and included staff with 
varied roles in the school. The leadership teams at these schools comprised a variety of staff, 
including teachers (three schools), deans (three schools), coaches (two schools), external 
support providers (one school), and community school liaisons (one school). For example, 


30 For example, survey measures used by Louis et al. (2010) include these: "teachers have an effective role in 
school-wide decision-making" and "the school's principal ensures wide participation in decisions about school 
improvement," both of which correspond to indicators on our index of distributed leadership. 
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membership at one of the elementary schools included administrators, teachers, coaches, 
paraprofessionals, the technology specialist, and the school counselor. 

• Leadership team has a clear set of responsibilities. For example, at two of the schools with the 
most evidence of distributed leadership, the primary role of the leadership team was to develop 
school improvement plans and monitor progress against goals and plans. The team at one of 
these schools was involved in strategic planning through setting goals for the upcoming year and 
reviewing lessons learned from past years. A member of the leadership team said the team 
works to "establish policy, working with input from staff. [We] try to do forward planning for 
next year. We take a look at what is successful and not successful." At the five other schools, 
leadership team members reported that the team's primary role was to serve as a vehicle for 
communication and collaboration. In these schools, the leadership team members provided 
input to administrators based on their interactions with teachers and communicated 
information to teachers on behalf of administrators and the leadership team. 

• Leadership team has a decision-making role. In five of the seven schools, principals reported 
that in addition to providing input, the leadership team members play an active role in making 
decisions. The principal at one of these schools reported that the leadership team makes school- 
level decisions regarding curriculum, school climate, data, and discipline. The principal at 
another school reported that the leadership team makes school-level decisions in other areas: 
scheduling, structuring the extended day, developing afterschool programs, and planning 
schoolwide events. 

• Principal's decision-making process is inclusive of nonteaching staff. In all seven schools, 
nonteaching staff— whether or not they were on the leadership team— including assistant 
principals and coaches, reported that they have the ability to contribute to the decision-making 
process, often through leadership team meetings, committee membership and meetings, formal 
requests for information, or the principal's open door policy. The literacy coach at one school 
said, "One thing we have in place is a formal coaches meeting, which we didn't have last 

year.. ..We [coaches, assistant principal, and principal] really think about what makes most 
sense; bring teacher feedback together, [it is a] shared decision-making process." 

• Teachers provide input into the decision-making process. In six of the seven schools, teachers 
reported that they provide input into the decision-making process. A teacher at one of these 
schools noted, "They always ask us for our opinions... It's a schoolwide involvement and decision 
making." One teacher at another school noted, "It's not just top down but also bottom up." 
Another teacher at the same school elaborated: "I feel I can make decisions to provide input and 
have my ideas evaluated in a collaborative way." 

• Teachers have leadership opportunities. Although participation in an inclusive and empowered 
leadership team is an important aspect of distributed leadership, schools that embrace 
distributed leadership often extend leadership opportunities— whether formal or informal— to 
teachers. At six of the seven schools with the highest levels of distributed leadership, teachers 
reported that they have opportunities to be school leaders. The most commonly reported 
leadership opportunities for teachers included leading professional learning activities and 
teams, including grade-level data teams and professional learning communities. At one school, 
for example, teachers are reportedly appointed as leaders of subject-specific improvement 
teams and grade-level team leaders. 
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Exhibit 3.2. 

Measure of Distributed Leadership in Core Sample Schools, 2010-11 



Respondents 
Reported 
Leadership 
Team Is Open 
and Varied 

Respondents 
Reported 
Leadership Team 
Has a Clear Set of 
Responsibilities 

Respondents 
Reported Leadership 
Team Has a 
Decision-Making 
Role 

Respondents Reported 
Principal's Decision- 
Making Process Is 
Inclusive of Non- 
teaching staff 

Respondents 
Reported Teachers 
Provide Input into 
the Decision- 
Making Process 

Respondents 
Reported Leadership 
Opportunities for 
Teachers at the 
School 

Overall 

Classification 

Blizzard Bay Elementary 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

High 

Gale Secondary 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

High 

Haven Way Elementary 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

High 

Inner Brooks High 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

High 

Island Bay Elementary 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

High 

McAlliston High 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

High 

Sterling Slope Elementary 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

High 

Aerovista High 

Yes 

Yes 

No 

Yes 

No 

Yes 

Moderate 

Baltimore Bridge Elementary 

No 

Yes 

No 

No 

Yes 

Yes 

Moderate 

Big Acorn High 

Yes 

Yes 

No 

Yes 

No 

No 

Moderate 

Elmsville High 

Yes 

Yes 

Yes a 

No 

No 

Yes 

Moderate 

Gillepsie High 

Yes 

Yes 

No 

No 

Yes 

Yes 

Moderate 

Meribel High 

Yes 

Yes 

No 

Yes 

Yes 

No 

Moderate 

Paul Bunyan High 

Yes 

Yes 

No 

No 

Yes 

No 

Moderate 

Peregrine Hill Elementary 

Yes 

Yes 

No 

No 

Yes 

Yes 

Moderate 

Proctor Point High 

Yes 

No 

No 

No 

Yes 

Yes 

Moderate 

Rossignol Elementary 

Yes 

Yes 

No 

No 

No 

Yes 

Moderate 

Tyron Elementary 

Yes 

Yes 

No 

Yes 

No 

Yes 

Moderate 

West Marble High 

Yes 

Yes 

No 

No 

No 

Yes 

Moderate 

Coral High 

Yes 

No 

No 

No 

No 

No 

Little/No 

Melon Elementary 

Yes 

No 

No 

No 

No 

Yes 

Little/No 

Raven Ridge Elementary 

Yes 

No 

No 

No 

No 

Yes 

Little/No 

Sawbuck Elementary 

Yes 

No 

No 

No 

No 

Yes 

Little/No 

Sherbrooke Elementary 

Yes 

Yes 

No 

No 

No 

No 

Little/No 

Tyro Trail Elementary 

Yes 

Yes 

No 

No 

No 

No 

Little/No 


Source: SST respondent interviews and focus groups, spring 2012. 

Notes: Includes 25 core sample schools. All school names are pseudonyms. 

a There was disagreement between respondents in this school for this measure. The principal reported that the leadership team members are the decision makers, but an 
instructional coach disagreed, noting that the "principal is the one that makes the decisions. He makes the decisions, and he tells us this is what needs to happen, and we just 
make it happen." 
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Respondents at 12 of the 25 core sample schools reported structures and opportunities for a 
moderate level of distributed leadership during Year 2 of SIG. In most of these schools (11 of 12), 
respondents' perceptions of staff involvement in the decision-making process were mixed. For example, 
the assistant principal at one elementary school described a "shared leadership model" with some 
infrastructure (such as the leadership team) to support it, but other school respondents (e.g., teachers) 
felt that the principal was still working on sharing decision making with a wide range of staff. Although 
certain staff members reported that the principal made decisions with input from others and that the 
school had a leadership team with a clear set of responsibilities, teachers did not uniformly report that 
they had input into decision making at the school. Flowever, teachers did report that they had 
opportunities to take on leadership roles. 

Respondents at 6 of the 25 core sample schools reported few structures and opportunities for 
distributed leadership during Year 2 of SIG. Although each of these schools had a leadership team that 
included nonadministrators, respondents did not indicate that the schools had more than one other 
feature. According to coaches and a teacher at one such school, the leadership team's role had 
diminished greatly under the new principal, who replaced an interim principal at the start of the 2012- 
13 school year. Respondents at two other schools— teachers in one school and coaches in another— 
reported that the leadership team was disorganized and meetings were often canceled or irregular. 

Respondents at these six schools did not perceive the principal as being collaborative or making 
decisions with input from teachers and other school staff. Likewise, teachers did not describe 
opportunities to provide input into the decision-making process. A coach at one school said, "You're to 
be seen and not heard." Another nonteaching staff member expressed similar feelings: "They [school 
leadership] hold meetings and include us but really aren't taking into consideration whatever 
suggestions [we make]." A teacher reported that "teacher input is not welcomed. If administration does 
not come up with the idea, it is not a good idea unfortunately." Only two respondents at this school, 
both teachers, said that the principal welcomed input, but one of those teachers immediately qualified 
his response by saying, "but some people think she's not open at all." 

In most of the core sample schools (15 of 25), respondents reported that administrators made an 
effort to establish structures to support distributed leadership during the first two years of SIG. Most 
of these schools were categorized above as having distributed leadership (seven schools) or having a 
moderate level of distributed leadership (seven schools) in Year 2 of SIG. One school was categorized as 
having little/no distributed leadership. Although distributed leadership strategies differed by school, 
principals reportedly promoted distributed leadership often through the creation of new teams or 
committees, the addition of new staff to an existing team (e.g., the leadership team), leadership 
opportunities for teachers and coaches, and additional ways for school staff to provide input into the 
decision-making process. 

In most of these cases, a new principal reportedly brought the shared leadership approach to the school. 
The school improvement team and school-based instructional specialist at one school described the 
principal, new to the school during the 2009-10 school year, as radically changing what used to be a 
confrontational relationship between teachers and school leadership to one of collaboration and open 
communication. The principal reportedly continued to develop leadership opportunities throughout his 
tenure, focusing on teacher-led professional learning opportunities during the 2010-11 school year. 
Similarly, the principal at another school reportedly brought about more distributed leadership when 
she started at the school during the 2009-10 school year and continued to develop capacity during the 
first two years of SIG. The principal said, "I wanted leadership to understand that they now needed to 
take on the role of building a strong staff.. .building capacity in leadership. We're making this decision 
collectively. It's no longer my decision. It's our decision." 
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In the remaining 10 schools, respondents reported that administrators did not develop distributed 
leadership during the first two years of SIG. Even though leadership was at least moderately distributed 
in five of these schools, respondents did not report that the principal took intentional actions to create 
new structures or opportunities to support distributed leadership. In the other five schools, there were 
few existing distributed leadership structures and no evidence of efforts to build such structures. For 
example, respondents described the principal at one school in 2011-12 as a top-down leader who 
reversed the work the interim principal had done to distribute leadership. 

Chapter Summary 

This chapter explored two leadership strategies that could potentially influence efforts to increase 
human capital in low-performing schools. The first— required by the SIG turnaround and transformation 
models— involved replacing principals. If the principal is replaced by someone who brings experience 
and expertise that is well suited to the school's needs, this mechanism has the potential to build the 
human capital of the school. Among the 25 core sample schools, it appears that district officials tried to 
hire principals with at least some relevant experience, and in half of the schools, respondents did 
describe the new principal as an improvement over the previous principal. We also found that in nine 
schools, continued leadership turnover was not infrequent, either because the first new hires proved to 
be a poor fit for the school or because school leaders left their positions for personal reasons or for 
other opportunities. 

The second capacity-building approach we examined is cultivating the leadership of other staff, 
including teachers and other nonadministrative staff at the school. Our site visit data suggest that 
principals at most of the core sample schools involved their staff in several features of distributed 
leadership in Year 2 of SIG, such as developing a leadership team and involving them in decision making. 
Moreover, respondents in 15 of the 25 core sample schools described efforts to enhance distributed 
leadership during the first two years of SIG. Although it was beyond the scope of this study to conduct 
an in-depth analysis of the quality of these interactions, respondents from these schools indicated that 
they welcomed the opportunity for increased professional engagement and leadership opportunities. 
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Chapter 4: Building Human Capital Through 
Staffing Decisions 


In the previous chapter, we examined how the 25 core sample schools strove to build human capital by 
replacing the principal and by distributing leadership amongst a wider range of staff. In this chapter, we 
examine two parallel strategies for teachers and other school personnel— namely, efforts to build 
human capital by (1) replacing or reassigning teachers and (2) hiring additional staff to provide various 
instructional or support functions in the school. 

As an improvement strategy, teacher replacement rests on assumptions similar to those for principal 
replacement: that many individuals employed in persistently low-performing schools lack knowledge, 
skills, or motivation needed to dramatically alter school performance, and that replacing them with 
more qualified individuals is necessary for improvement. These assumptions are reflected in the SIG 
turnaround model, which requires schools to replace at least 50 percent of their teachers. Classroom 
teachers are not the only school personnel who perform essential functions. Other instructional and 
support staff, such as coaches, also may perform functions that are necessary for improvement. Thus, a 
second improvement strategy is to bring in additional staff who can provide these supports. To assist 
with implementing this strategy, SIG allows schools to use the grant money to fully or partially support 
salaries for staff members whom schools may not have otherwise been able to hire, or whom might 
have been otherwise laid off due to budget reductions (U.S. Department of Education, 2010). 

Building a skilled and motivated staff at low-performing schools can be challenging. The school 
environment or district policies may make it difficult for principals to recruit and retain the personnel 
they need (Guarino, Santibanez, & Daley, 2006). New staff may not have greater skills, or they may be 
stronger in some areas (e.g., motivation) but weaker in others (e.g., pedagogical skills). Even if new hires 
are improvements over previous staff, large-scale changes such as those called for in the turnaround 
model may exacerbate existing patterns of instability and high teacher turnover. Thus, (1) how 
challenging the context for a school is with regard to staff recruitment and retention, (2) how supportive 
the district is in this area, and (3) how well school leaders are able to navigate these challenges and 
supports in a purposeful way to meet their unique needs or goals, are important corollaries to these two 
improvement strategies. We therefore also examine these three issues in this chapter. 
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Box 4.1. Key Chapter 4 Findings 

• About half of the core sample schools (12 of 25) replaced at least 50 percent of their teachers 
between 2009-10 and 2011-12. Respondents at 7 of these 12 schools perceived the 
replacement as positive for the school, bringing new energy and improved morale. 

• During the first two years of SIG, almost all schools (24 of 25) created new nonteaching 
positions, most often coaches (instructional, technology, data) and school administrators. 

• Principals in about half of the schools (12 of 25) indicated that staffing decisions were made 
with specific school needs or goals in mind. 

• Principals or district officials in 18 of 25 core sample schools reported that recruitment and/or 
retention challenges limited the school's ability to build a skilled and motivated staff. District- 
level conditions and policies (such as layoffs, involuntary transfers, and hiring processes that 
limited teacher applicant pools) were the most frequently cited recruitment or retention 
challenges (12 of 25 schools). 

• Principals and district officials in nine schools across seven districts reported that their district 
provided advantages to SIG schools in the hiring process (such as the ability to screen and 
interview candidates before other schools) to better enable them to recruit qualified staff. 
Teachers in eight schools across four districts reportedly received a monetary bonus for 
working at the school. 


Replacing Teachers 

A central performance problem for some persistently low-performing schools appears to be the quality 
of the teaching staff (Aladjem et el., 2010; Herman et al., 2008; Max & Glazerman, 2014). If so, then 
replacing the staff may be necessary to enact meaningful change that cannot be achieved solely through 
professional learning for existing teachers. While efforts to replace teachers are not uncommon in low- 
performing schools (De la Torre et al., 2013), persistent staff instability over time— whether intentional 
or due to other factors— may be detrimental to a school (Ronfeldt, Loeb, & Wyckoff, 2013). Replacing 
some teachers may be perceived as necessary to realize the principal's vision for the school, but large- 
scale changes could be disruptive, especially when combined with leadership changes (for example, as is 
prescribed by the SIG turnaround model). If there are also challenges in recruiting and retaining new 
staff, building capacity may become even more difficult. We begin our analysis by examining the 
prevalence of teacher replacement in the core sample schools (see Box 4.2), and then move to the 
perceptions, challenges, and supports related to this improvement strategy. 

About half of the core sample schools (12 of 25) replaced at least 50 percent of their teachers 
between 2009-10 and 2011-12. Two schools replaced their teachers one year prior to SIG (2009-10), 
nine schools did so in the first year of SIG (2010-11), and two schools did so in the second year of SIG 
(2011-12). One of these schools replaced at least 50 percent of their teachers in both Years 1 and 2 of 
SIG. Among these 12 schools, 9 were implementing the SIG turnaround model, which requires that at 
least 50 percent of the teachers be replaced. 31 The other three schools adopted the restart or 


31 Schools implementing the SIG turnaround model were required to replace at least 50 percent of their teachers in 
the first year of the grant. Six of the nine turnaround schools in our core sample did so in the first year. Two 
schools encountered a delay in receiving SIG and did not replace staff until the second year. The one remaining 
school replaced its teaching staff just before the first year of SIG. 
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transformation models, which did not require teacher replacement. The charter management 
organizations for the two restart schools hired entirely new teaching staffs in the first year of SIG. The 
principal at the one transformation school explained that he was seeking to build a teaching staff that 
was committed to his new school improvement goals. 


Box 4.2. Teaching Staff Stability in Core Sample Schools 

School classifications on teaching staff stability are based on Year 1 and 2 interview and focus group 
data (see Exhibit B.4 for more detail on the analytic procedures). For this analysis, respondents refer 
to individuals from the following respondent groups: district administrators, principals, teachers, and 
instructional coaches. 

Greater than half of teachers each year 

• At least two respondents reported that at least 50 percent of the teaching staff was new at 
the beginning of both Years 1 and 2 of SIG. 

Less than half of teachers in one year, and more than half of teachers in the other year 

• At least two respondents reported that at least 50 percent of the teaching staff was new at 
the beginning of Year 1, but not Year 2, or vice versa. 

Less than half of teachers each year 

• At least two respondents reported that less than 50 percent of the teaching staff was new at 
the beginning of both Years 1 and 2 of SIG. 


Perceptions of Teacher Replacement 

The perceptions of school stakeholders matter in the improvement process. Attitudes about new 
teachers and the actions of the administration to bring them to the school are likely to influence the 
degree to which remaining teachers collaborate and support the new teachers, and how supportive they 
are of the reform effort in the school more generally. To better understand whether large-scale teacher 
replacement was perceived as positive for the school and its improvement trajectory, we analyzed 
interview data from the 11 core sample schools that implemented teacher replacement in 2010-11 or 
2011-12 (see Box 4.3). Respondents shared their perceptions of the quality of new teachers, as well as 
their perceptions of the teacher replacement process (i.e., the rules that govern adding and removing 
staff, together with the extent to which principals and teachers can make decisions about staff 
placements). 

Among the 11 schools that replaced teachers between 2010-11 and 2011-12, respondents in 7 
schools (4 turnaround, 1 transformation, and 2 restart 32 ) characterized the addition of new teachers 
as positive for the school, bringing new energy, improving morale, and creating the potential for higher 
teacher capacity. In three others (all turnaround), respondents characterized the teacher replacement 
as neutral, just "another instance of change." For the one remaining turnaround school, respondents 
indicated that the teacher replacement process was demoralizing and aggravated problems for the 
school (Le Floch et al., 2014). 


32 At the two restart schools, all teachers, including the ones interviewed for this study, were new to the school at 
the time of data collection. These teachers could not speak to the previous year or change from the previous year, 
but they were able to speak to their perceptions of the quality of the school's current staff. 
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Box 4.3. Perceptions of the Teacher Replacement Process 

School classifications on perceptions of the teacher replacement process are based on interview and 
focus group data in Year 1 for nine schools (Le Floch et al., 2014) and in Year 2 for two additional 
schools that replaced their teachers in the second year of SIG (see Exhibit B.5 for more detail on the 
analytic procedures). For this analysis, respondents refer to individuals from the following respondent 
groups (excluding newly hired staff): district administrators, principals, and teachers. 

Positive 

• At least three respondents indicated that the new teachers introduced as part of the 
replacement process were beneficial (i.e., "bringing new energy," improving staff morale, 
increasing teacher quality); AND 

• No respondent described the teacher replacement in terms such as "biased" or "unfair." 

Neutral 

• Respondents did not comment on the quality of the teacher replacement process or the 
quality of new teachers, or described the process in neutral terms, such as "another instance 
of change." 

Negative 

• At least three respondents indicated that the new teachers introduced as part of the 
replacement process were detrimental to the school (i.e., weakening staff morale, decreasing 
teacher quality); OR 

• At least three respondents described the teacher replacement process in terms such as 
"biased" or "unfair." 


In one school implementing the turnaround model where the replacement process, which was delayed 
until 2011-12, was categorized as "positive," the principal explained that the school looked for new 
teachers with the "right mindset" and the ability to set "high expectations for students." Hiring 
experienced staff was another priority according to the principal, who explained that all new teachers in 
2011-12 brought three to five years of teaching experience. He expected that with this new staff, the 
school's data would "speak for itself," and that on balance, the experience had "absolutely" been 
positive. Three out of four teachers interviewed echoed his comments, saying, for instance, that new 
teachers "really want to make a change and be here." Similarly, respondents at another turnaround 
school reported that bringing in new teachers was beneficial to the school, like a "rebirth." At a third 
school, the principal, teachers, instructional coaches, and parents universally characterized the new 
teachers as motivated and skilled, and indicated that the new teachers improved the school climate and 
instructional quality. 

In contrast, respondents at another turnaround school that replaced teachers in 2011-12 appeared to 
have mixed perceptions about the change. Few respondents at this school offered comments about the 
teacher replacement process but had more to say about the quality of the new teachers. One 
instructional coach felt that the new staff were helping to "change the culture of the school" and were 
"more willing to examine their own instructional practice." However, the principal spoke less positively 
about the capacity of the teachers (both new and existing). The other two schools where respondents 
expressed neutral sentiments overall (both of which replaced teachers in 2010-11) were located in a 
district where schools implementing the turnaround model reportedly replaced staff by shuffling 
existing teachers around to different schools— a process referred to as a "lemon dance." In this district, 
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principals reportedly selected new teachers from an applicant pool that included teachers who had been 
removed or voluntarily transferred from other schools in the district. 

Finally, respondents at one turnaround school that replaced teachers in 2010-11 described the process 
as "demoralizing." Although new Teach for America teachers who had been added were perceived as 
being "bright and motivated," the process of identifying teachers to be replaced was perceived as 
biased. Teachers described classroom observations as subjective, and staff had mixed views about the 
resulting group dynamic among new and returning staff. Flere, the replacement process was reportedly 
conducted in a "toxic" manner that may have limited efforts to build human capital. 

Creating Nonteaching Positions 

As an improvement strategy, creating new nonteaching positions could help build human capital in a 
few ways. It may free teachers from responsibilities that they otherwise would have to take on and 
allows them to focus their attention more on tasks directly related to instructing their students. It may 
allow the school to do additional kinds of capacity-building work— for example, coordinating 
partnerships with community groups or educating parents. It may also be a way to provide teachers 
with targeted, capacity-building support for their instruction— for example, hiring English language 
learner coaches and subject-matter coaches. Although funding positions with soft money might pose 
risks for sustainability, schools could potentially seek other funding to maintain newly-added staff. 

Most core sample schools (24 of 25) reported creating new nonteaching positions at some point 
during the first two years of SIG. 33 About half of the schools (13) reported adding new nonteaching 
positions in both years. The remaining 11 schools reported adding new nonteaching positions in only 
one year (2010-11 or 2011-12). Schools that added new nonteaching positions in either year may or 
may not have continued that position in later years. The types of positions added varied by school with 
coach positions— including instructional coaches, data coaches, and technology coaches— being the 
most commonly reported (14 schools). Respondents frequently reported that these positions were 
added to build the skills of teachers in content and pedagogy. For example, one school reported adding 
four new coaches during the first year of the grant (2010-11). The principal explained that he told the 
district that he needed full-time "in-house" coaches to be successful. Fie reported that these coaches 
built capacity by working with teachers and planning strategies for meeting school goals. At another 
school, the principal explained that although a technology coach position was not originally in the SIG 
budget, he used money originally allocated for a curriculum coach to hire a part-time coach for the 
2011-12 school year to build the technological skills of school staff. The principal said it was a "huge 
need" because "professional development on technology is really lacking." A member of the school's 
governance board agreed, commenting that "[the technology in classrooms] is not being utilized 
correctly." 

Respondents at 11 schools also reported adding new administrators, such as assistant principals. At one 
school, long-term temporary administrators were assigned to work at the school both to receive training 
on how to become effective school administrators and to support the school principal in carrying out his 
goals. After filling one of these temporary positions, educators would be given permanent assistant 
principal positions at other schools within the district. Other schools also added other nonteaching 
positions, including parent or community liaisons, social workers, and technology coordinators. For 


33 Identified new nonteaching positions are those reported by at least one of the following respondents: district 
administrator, principal, teacher, instructional coach, or school improvement team member (see Exhibit B.6 for 
more detail on the analytic procedures). 
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example, the principal at one school decided to use some of his SIG funding to add a full-time parent 
liaison to the school staff. He justified this position by explaining that the liaison worked closely with 
parents to help them learn English. Doing so, he argued, improved kindergarten readiness at his school. 
In another district, SIG schools reported hiring community liaisons to coordinate activities and build 
relationships with parents and community-based organizations. 

Respondents indicated that SIG was often used to fully or partially support the salaries of staff, whom 
schools may not have otherwise hired or whom would have been otherwise laid off as a result of budget 
cuts. At one school, the district used SIG to hire a social worker to focus on the needs of the school's 
homeless student population. According to respondents, this position would not have been possible 
without SIG. 

Purposeful Approaches to Staffing: Addressing School Needs 
or Goals 

We hypothesize that staffing changes such as those previously discussed in this chapter are most likely 
to build human capital when they address identified school needs or goals. If principals can identify 
school needs and meet those needs by strategically hiring new staff or adapting the responsibilities of 
existing staff, then these staffing efforts may be more likely to contribute to school improvement. Using 
qualitative data from 2011-12, 34 we examine the extent to which principals in our core sample schools 
were able to implement staffing strategies in a purposeful manner; that is, one in which principals 
provided evidence of a pattern of intentional decisions related to hiring new staff or making changes to 
the roles and responsibilities of existing staff that aligned with specific school needs or goals. Our 
analyses take into consideration the broader district context that may limit a school leader's ability to 
implement desired changes related to hiring and firing staff. Principals who were aware of these district 
limitations but still proactive in their efforts to adopt purposeful staffing strategies to the extent 
possible were classified as being purposeful (see Box 4.4). 


Box 4.4. Principal's Approach to Staffing Decisions in Core Sample Schools 

School classifications on the purposefulness of the principal's staffing approach are based on Year 2 
principal interview data (see Exhibit B.7 for more detail on the analytic procedures). Data from other 
school-level respondents, including teachers and instructional coaches, provided details about the 
principal's approach to staffing decisions. 

Purposeful approach to staffing decisions 

• The principal clearly articulated how staffing decisions addressed school goals or needs (e.g., 
removing a teacher due to poor performance and replacing her with someone with the 
"right" skill set and strong motivation, or reassigning teachers to different grades to better 
align their skills to students' needs; creating an additional instructional coach position to give 
teachers "the support that they need"). 


34 These analyses rely primarily on Year 2 interview and focus group data (2011-12), which incorporated questions 
specifically about capacity-building efforts at the school and district levels. In cases where we had additional data 
on this topic from 2010-11, we included it. In 2010-11, interviews and focus groups concentrated on identifying 
each core sample school's performance problem as well as how SIG money would be used to drive school 
improvement; these discussions sometimes included data on staffing strategies. 
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Box 4.4. Principal's Approach to Staffing Decisions in Core Sample Schools 

(continued from previous page) 

No evidence of purposeful approach to staffing decisions 

• The principal clearly articulated how staffing decisions were not aligned with school goals or 
needs (e.g., accepting low-quality teachers due to district mandate); OR 

• The principal did not provide any evidence on how staffing decisions addressed school goals 
or needs (e.g., replacing teachers leaving for personal reasons such as retirement or 
relocation). 


Principals at about half of the core sample schools (12 of 25) provided evidence of a purposeful 
approach to staffing. For example, the principal at one of these schools explained that second-grade 
achievement gains "were so low" that she had to do something about it. She attributed these low scores 
to a second-grade teacher near retirement. To address the inadequate instruction provided by this 
teacher, the principal reorganized her existing teachers by moving two first-grade intervention teachers 
into the second-grade instruction positions, a decision she felt would strengthen the quality of teaching 
in that grade. She also moved the problematic second-grade teacher to a third-grade teaching position 
because he had more experience teaching third grade. She then explained how she used the district's 
mentor review and collaborative process to try to improve the teacher's instruction. Ultimately, the 
principal was able to counsel the struggling teacher into retirement midway through the year and hire 
someone she perceived to be of higher quality. To accomplish these staffing changes, the principal 
worked with district administrators who supported her staffing plan, but union agreements limited their 
ability to remove low-performing teachers quickly. While the district worked to establish a 
memorandum of understanding to facilitate easier staffing changes at SIG schools, the principal created 
and implemented purposeful strategies to minimize the impact of less effective staff. 

At another school, the district was also open to working with the principal as she tried to execute her 
desired staffing strategies. Flere, as in most districts, existing policies place some limits on schools with 
respect to staffing. One district administrator explained that the balance between existing policies and 
exceptions for SIG schools is "always being renegotiated." Working within this context, the principal 
interpreted data on student behavior to identify weak instructors. "When we looked at our behavior 
data and our spikes, we knew it. It was our elective teachers." The principal removed those teachers, 
including an art teacher and a music teacher, and replaced the art teacher with a science teacher 
because of the school's STEM (science, technology, engineering, mathematics) focus. She also 
capitalized on many of the existing teachers' probationary status to remove and replace them with "rock 
stars" whom she personally assessed and recruited. 

At some core sample schools, however, the district context reportedly created many more limitations on 
principals' abilities to implement staffing strategies. Principals in these schools may have had purposeful 
approaches to staffing but found it very difficult to implement them. At one such school, district budget 
cuts were greatly affecting the school. The principal explained, "If I don't have the SIG grant, I'm very 
concerned about the sustainability of my school.... Without that... I wouldn't be able to have enough 
teachers to teach. So I've been very forthright with the district to say this is a problem." Fie went on to 
describe additional outcomes of the budget crisis, including that the instructional coaches he hired were 
forced to provide direct instruction as classroom teachers for parts of each day, thereby limiting their 
availability to work with teachers. Despite these constraints, this principal was still able to implement 
some of his desired staffing changes. For example, he terminated several staff members during the year 
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because of poor performance. His assistant principal confirmed that they had "moved [a] tremendous 
amount of people out" who were not the right fit for the building. 

Principals at the remaining 13 schools did not provide evidence of a purposeful approach to staffing. 

Some of these principals described district policies that limited their ability to make intentional staffing 
decisions, as was the case with some of the schools previously discussed. However, unlike those schools, 
principals at these schools seemed unable to execute purposeful staffing strategies within such a 
constrained environment. For example, the principal at one school reported a staffing environment that 
did not address school needs: District layoffs in 2011-12 reportedly caused the school to lose quality 
teachers, and the school has historically had to hire teachers who were "left over" in the applicant pool 
because teachers prefer to work at higher-performing schools. This principal did not describe trying to 
maximize the staff to best support the students, nor did he describe any efforts to remove these 
undesirable hires. 

Principals from other schools in this category did not describe a need to change staff. For example, one 
principal indicated that the applicant pool was full of "really qualified people," adding that the school 
retained more than 90 percent of its staff from the prior year. Although he did not provide evidence of 
an intentional approach, he also did not report a need for changes to the school's staffing strategy. 

Perceived Recruitment and Retention Challenges 

Respondents in our core sample schools reported various challenges in trying to recruit and retain a 
stable group of skilled and motivated staff. Their recruitment and retention challenges are consistent 
with those identified in existing research, such as difficult work environments (Barry, Raspberry, & 
Williams, 2007), stringent district hiring policies (Wayne et al., 2011), and rural settings (Rosenberg et 
al., 2014). For example, our site visit data reveal that when replacing teachers, many principals reported 
being unable to request the departure of specific teachers. Union contracts required that principals ask 
for voluntary transfers, and if more transfers were required, the teachers with the lowest seniority, 
irrespective of quality, were transferred to other schools. 

Perceived teacher retention and recruitment challenges are defined as those for which the principal 
and/or a district administrator described associated challenges based on Year 2 interview data (see 
Exhibit B.8 for more detail on the analytic procedures). 35 Exhibit 4.1 identifies perceived teacher 
recruitment and retention challenges across the 25 core sample schools. Although this analysis focuses 
on teacher recruitment and retention, some of the same challenges likely apply to the recruitment and 
retention of other types of staff as well (e.g., coaches, administrators). 

Overall, we found that school respondents reported facing a number of recruitment and retention 
challenges in 2011-12. Conditions associated with the district context, such as widespread teacher 
layoffs and policies that required SIG schools to staff specific positions occupied by previously-removed 
teachers rather than have the flexibility to choose teachers independently, were most frequently 
discussed by respondents. 


35 Although we recognize that many of the challenges identified by respondents may not be unique to this time 
frame, we included teacher recruitment and retention as a key focus of Year 2 data collection efforts. Challenges 
related to recruitment and retention raised by respondents in Year 1 were identified and taken into consideration 
as site visitors prepared for Year 2 site visits. 
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Exhibit 4.1. 

Perceived Teacher Recruitment and Retention Challenges, 2011-12 
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Source: SST respondent interviews and focus groups, spring 2012. 

Notes: Includes 25 core sample schools. Categories are not mutually exclusive, except for No challenges reported. 


Principals and district officials at 18 of 25 core sample schools reported that recruitment or retention 
challenges limited the school's ability to build a skilled and motivated staff. Teacher recruitment (but 
not retention) was reported as a challenge at 3 of these schools, teacher retention (but not recruitment) 
at 11 schools, and both teacher recruitment and retention at 4 schools. 

Principals and district officials at 12 of these 18 schools reported that district conditions contributed 
to staffing challenges. These conditions included districtwide budget cuts and layoffs, limits on the pool 
of teachers from which principals could hire, and involuntary transfer procedures. Principals and district 
officials at 10 of these 12 schools reported that districtwide budgetary constraints forced teacher 
layoffs. Teachers were reportedly "pink slipped" even when school administrators preferred to keep 
them. For instance, respondents at one school indicated that several newer teachers (many of whom 
had started during the first year of SIG) lost their positions in Year 2 as a result of districtwide teacher 
layoffs. The principal explained that two and a half weeks before school started, the school lost 24 staff 
members, including "a lot of people that truly wanted to be here and that had built relationships with 
the children." 

Principals and district officials at 7 of these 12 schools reported that the district's applicant pool included 
teachers who had been removed from other schools, or those who were "left over" among the teachers 
not yet hired by other schools. The principal and external provider at one such school described 
recruiting "competent" teachers as a challenge. The principal explained: 

Yes, we were given the opportunity to hire people, but the people come off of a list, and the list 
may not necessarily have the best candidates. Also, when you look at the teaching staff, there's 
no flexibility at all. You can't bring anyone in from outside the district. And who you're 
interviewing off that placement list are people that need to be placed, and they need to be 
placed because they have been displaced from someplace else that didn't want them in their 
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place. And 9 times out of 10, I don't want them in my place either.... We are moving the same 
incompetent people around to different schools. 

Principals and district officials at 3 of these 12 schools reported that they had no choice but to accept 
teachers who had been involuntarily transferred to the school. The principal at one such school 
explained that he received an involuntary transfer who was not wanted in any school in the district. This 
teacher was described as "a bad teacher" who had been disruptive at another school in the district and 
was a "toxic" influence. 

Of these 12 schools with reported district-level challenges, 2 also reported that the district took actions 
to help schools overcome these challenges. The principal at one such school explained that although 
schools in the district are required to hire from a pool of teachers laid off from other schools in the 
district, SIG schools are able to interview teachers from the pool, rather than having "to take the next 
person off the layoff list." A district respondent for another one of these schools indicated that SIG 
schools are given preference in external hiring and protected from administrative transfers, to the 
extent possible. The principal confirmed that he is given the opportunity to hire externally before other 
non-SIG principals, although he did also report that other district staffing policies posed challenges for 
the school. 

Principals and district officials at 7 of the 18 schools with recruitment or retention challenges reported 
that school-level constraints— such as poor school reputation, stressful school environment, or long 
commutes to school— contributed to the challenges. Principals and district officials at four of these 
schools reported that the school's reputation contributed to challenges attracting and retaining staff. 

For example, one of the principals noted that the school's reputation as a "tough school" made 
recruiting staff challenging, explaining that people who are not familiar with the school are afraid to 
work there. 

Principals and district officials at three schools reported that a stressful school environment contributed 
to challenges attracting and retaining teachers. The principal at one such school reported that the 
students in third grade were very difficult, a perception supported by the school's teachers and coaches. 
Teachers explained that, as a result, the school struggled to keep third-grade teachers for even one full 
school year, and that the school had lost seven third-grade teachers in the middle of 2011-12. 

Principals and district officials at four schools reported that teachers' long commutes to school 
contributed to challenges attracting and retaining teachers. The principal at one such school noted that 
the remote location made the drive "brutal," a sentiment that was echoed by teachers, the district 
official, the external provider, and the union representative. 

Principals and district officials at 7 of the 25 core sample schools indicated that neither recruitment 
nor retention presented a challenge for the school in 2011-12. These schools appeared to have stable 
teaching staffs that year. Principals in these schools reported that fewer than 10 percent of teachers 
were new to the school in 2011-12. Two of them had replaced 50 percent or more of their teachers in 
2010-11. Both of these schools were implementing the restart model under SIG and replaced all staff 
when they were handed to charter management organizations in the first year of SIG. The principals at 
these schools reported successfully retaining the majority of their new teachers, limiting the need for 
recruitment in 2011-12. 

Four of the seven schools without reported recruitment or retention challenges in 2011-12 received 
"high" ratings in 2010-11 on our organizational capacity measure, which included components related 
to teacher collaboration, safe and orderly environment, and teacher-teacher trust. These components of 
organizational capacity could improve a school's ability to retain existing staff, thus reducing the need to 
recruit new staff (Le Floch et al., 2014). For example, one of these schools reportedly had a stable staff 
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between 2010-11 and 2011-12 (less than 10 percent teacher turnover), and the principal reported no 
trouble with recruitment, explaining, "We've been having really good results with people applying to 
positions from all over. We've had a great applicant pool. It's been really positive; really qualified people 
want to work with us." This school is situated in a "benign" context, and school respondents described 
the culture of the school in positive ways. In addition, the school is located in a state where layoffs have 
occurred consistently for the past few years, so many teachers were reportedly eager to find work. All 
three factors could explain why the applicant pool at this school has been strong and plentiful. Similarly, 
the principal and district officials at another school reported that a weak economy meant that teachers 
who had jobs wanted to keep them, and that lots of people competed for any available jobs. There were 
always numerous applicants for open positions at the school, according to the district administrators, 
union representative, and principal. 

Structures to Support Staff Recruitment and Retention 

Research has identified a variety of structures that can be used to support teacher recruitment and 
retention (Guarino, Santibanez, & Daley, 2006). In this section, we describe how districts were using 
specific organizational structures to address recruitment and retention challenges reported by core 
sample schools. Identified supports are defined as those described by the principal and/or a district 
administrator based on Year 2 interview data (see Exhibit B.9 for more detail on analytic procedures). 36 

Principals and district officials in nine schools across seven districts reported that their districts 
provided advantages to SIG schools in the hiring process to better enable them to recruit qualified 
staff. For example, one district official explained (and was corroborated by one of the school principals) 
that SIG schools did not receive forced-transfer teachers in core subject areas. A district official in 
another district reported that principals of Title I schools (including the one in our core sample) were 
able to screen and interview candidates for open positions before other schools in the district. These 
types of advantages may have helped schools secure the caliber of staff they needed to enhance their 
ability to address the needs of their students. 

Teachers in eight schools across four districts reportedly received a monetary bonus for working at 
the school . 37 The decision to provide bonuses using SIG was driven by the principal in one of these eight 
schools. In the other seven— which were situated in three districts— the bonuses were part of a district 
initiative. In one of these districts, SIG was used to provide bonuses for teachers in all SIG schools. The 
other two districts had established bonuses, unrelated to SIG, for teachers in a set of low-performing 
schools that included some SIG schools. Some bonuses were paid as signing bonuses, and others were 
paid at intervals throughout teachers' tenure at the school. For example, teachers at a set of low- 
performing schools in one district received bonuses at the end of their first year, fourth year, and sixth 
year teaching at the school. 

Principals in two schools reported extra supports for teachers to alleviate the expense of their long 
commutes to the school. One provided a gas stipend to teachers teaching in the remote community, and 
the other provided a van service from the communities in which teachers live to the school. 


36 Although we systematically collected information about these supports from respondents in Year 2, the supports 
they described were not necessarily new in Year 2 of SIG implementation. 

37 Other types of monetary benefits, such as performance-based incentives paid if certain student improvement 
goals are met and extra pay for working during extended hours or participating in additional professional learning 
opportunities, were not included in this analysis. 
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Chapter Summary 

Replacing teachers and adding new nonteaching positions are examples of staffing strategies schools 
may pursue in an effort to build human capital. All 25 core sample schools reported using some 
combination of these strategies to change the composition of their staff during the first two years of 
SIG. Specifically, 12 of the 25 core sample schools replaced at least half of their teachers, and 24 of the 
25 schools created new positions for nonteaching staff, such as administrators or instructional coaches. 
Some of these staff changes were perceived to build human capital, while others were perceived as 
creating "too much turnover." 

At 12 of our 25 core sample schools, principals provided evidence of a purposeful approach to staffing in 
2011-12. These principals reported being able to develop strategies that enabled the hiring of new 
nonteaching staff members or the replacement of old teachers in a manner that aligned with the 
school's overall needs and goals. District conditions and policies played a role in either supporting or 
challenging the schools' ability to recruit and retain teachers. Some districts reported creating systems 
and structures that facilitated hiring in SIG schools, for example, by providing monetary incentives for 
teachers who work at SIG schools. However, among the 18 core sample schools where the principal or 
district official reported difficulties in recruiting or retaining high-quality teachers in 2011-12, most 
described district conditions (e.g., budget cuts resulting in layoffs) or policies (e.g., limiting the teacher 
hiring pool) that hindered recruitment or retention efforts. 

Overall, the evidence from our core sample schools suggests that requiring the large-scale replacement 
of teachers over a short time period in low-performing schools (as is the case with the SIG turnaround 
model) may not be straightforward. Wholesale staffing changes may be necessary in schools with a 
dysfunctional or "toxic" culture or climate, but the changes are by nature disruptive. Whether such 
disruption will likely enhance human capital depends in part on schools' ability to recruit or retain the 
teachers that best "fit" with school needs. This in turn will depend on how able principals are to identify 
their school needs, how able they are to identify and remove teachers who are a poor fit, how qualified 
the teacher applicant pool is, and how much flexibility the principal has to hire teachers from the 
applicant pool. Our data suggest that a principal's own capacity and district policies may be important 
contextual factors that help determine what effects such requirements ultimately have in these low- 
performing schools. 
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Chapter 5: Building Human Capital Through 
Teacher Professional Learning 


Professional learning aims to build teachers' knowledge and skills and subsequently improve student 
outcomes. The SIG transformation and turnaround models require districts to provide their SIG schools 
with "ongoing, high-quality, job-embedded professional development that is aligned with the school's 
comprehensive instructional program and designed with school staff to ensure they are equipped to 
facilitate effective teaching and learning and have the capacity to successfully implement school reform 
strategies" (U.S. Department of Education, 2010, p. 36). Increasing professional learning opportunities 
was the most common improvement strategy reported in our core sample schools during the first year 
of SIG (Le Floch et al., 2014), and is consistently cited as an improvement strategy in low-performing 
schools more generally (Opfer & Pedder, 2011). 

While there is limited rigorous evidence demonstrating the effects of professional development (PD) on 
teacher and student outcomes (Campbell & Malkus, 2011; Garet et al., 2008; Garet et al., 2010; National 
Mathematics Advisory Panel, 2008; Wayne et al., 2008; Yoon et al., 2007), there are numerous 
theoretical frameworks and related suggestive evidence identifying desirable features of PD. These 
features include having a substantial number of PD hours (Desimone, 2009; Garet et al., 2001; Porter et 
al., 2003) and sustaining professional learning activities over a period of time rather than implementing 
them sporadically (Flawley & Valli, 1999; Opfer & Pedder, 2011). These basic features facilitate a 
stronger environment for professional learning that can focus deeply on content (Darling-Flammond et 
al., 2009; Kennedy, 1998); make connections with district and school improvement goals (Desimone et 
al., 2002a; Elmore, 1997); provide opportunities for teacher-centered "active learning" (Bransford, 
Brown, & Cocking, 2000); and create opportunities for teachers from the same school, department, or 
grade-level to learn collectively (Birman et al., 2000; Desimone, 2009; Desimone et al., 2002b; Garet et 
al., 2001). For example, professional learning communities— ongoing, knowledge-centered communities 
of practice— could potentially improve teacher knowledge, self-efficacy, and positive views of the school 
through processes of collective learning, decision making, and collaboration (DuFour, 2004a; Olivier & 
Hipp, 2006; Stoll etal., 2006). 

In this chapter, we examine a few of these features of professional learning in the context of the 25 core 
sample schools. First, we document the number of professional learning hours in which teachers 
reported participating. Second, we discuss the degree to which professional learning opportunities 
appeared to be purposeful or tied to school goals. Third, we examine the extent to which schools took 
steps to provide teachers with the opportunity to participate in job-embedded learning activities. 

Fourth, we examine the schools' efforts to establish structures that support teacher learning 
opportunities, such as strategies to foster collaboration among staff and structures designed to help 
teachers use data to identify student needs and differentiate instruction. Finally, we examine whether 
teachers reported changing their practices based on participating in professional learning opportunities. 
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Box 5.1. Key Chapter 5 Findings 

• Teachers' reported participation in professional learning opportunities during Year 2 of SIG 
(2011-12) varied across and within schools. The median number of hours in each core sample 
school ranged from 0 to 187, while the difference between the 25th and 75th percentiles 
within each school ranged from 36 to 158 hours. 

• More teachers reported participating in professional learning focused on math, literacy, and 
data use than on English language learner instruction, special education, or classroom 
management. 

• Principals and teachers in 6 of the 25 core sample schools articulated connections between 
professional learning activities and the school's goals or needs. 

• Teachers in most of the core sample schools (15 of 25) on average reported spending a larger 
proportion of hours in job-embedded professional learning activities (e.g., classroom 
coaching, structured common planning time, meetings with mentors, consultation with 
outside experts, observations of classroom practice) than in more traditional activities (e.g., 
workshops, conferences). 

• Interview respondents in most core sample schools reported developing organizational 
structures and systems to support teacher collaboration (16 of 25) and data use (19 of 25) 
during the first two years of SIG. 

• In 17 of 21 core sample schools with sufficient data, most teachers reported learning and 
changing their practice after participating in professional learning on math, literacy, or data 
use. 


Teachers' Participation in Professional Learning Opportunities 

A nationally representative study of teacher quality under the No Child Left Behind Act reported that 
teachers in Title I schools identified for improvement participated in an average of 111 hours of 
professional learning per academic year, including the summer, across any topic (Taylor et al., 2010). 
Other studies suggest that receipt of a substantial amount of PD— an average of 49 hours— on particular 
topics or strategies is associated with a 21 percentile point boost to student achievement (Yoon et al., 
2007). Because these two findings are based on a similarly broad and inclusive definition of professional 
learning used in our study, they offer useful benchmarks for our findings. 

Our core sample schools varied in the median number of professional learning hours reported by their 
teachers, ranging from 0 hours to 187 hours (see Exhibit 5.1). In 21 of the 23 schools with sufficient 
teacher survey data in spring 2012, at least 80 percent of teachers reported participating in at least 
some professional learning activities during the 2011-12 academic year. In 5 of these 21 schools, 100 
percent of teachers reported participating in professional learning activities. In one core sample school, 
most teachers reported participating in no professional learning. 

The amount of professional learning hours reported by teachers also varied within each of our core 
sample schools, with the difference between the 25th and 75th percentiles 38 within each school 
ranging from 36 to 158 hours (see Exhibit 5.1). For example, teachers at Meribel High reported a wide 


38 This is the interquartile range, a measure of the range between the upper quartile (the 75th percentile of 
reported PD hours) and the lower quartile (the 25th percentile of reported PD hours). 
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range of professional learning hours, with the 75th percentile corresponding to 220 hours and the 25th 
percentile corresponding to 62 hours. In contrast, teachers at Peregrine Elementary reported an upper 
quartile value of 60 hours and a lower quartile value of 21 hours. 


Exhibit 5.1. 

Number of Teacher-Reported Hours of Professional Learning in 2011-12, 
by Core Sample School 
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Source: SST teacher survey, spring 2012 

Notes: Includes 23 of 25 core sample schools (N = 810 teachers). Two schools were excluded from this analysis because they did 
not meet the 50 percent response rate threshold on the teacher survey. All school names are pseudonyms. UQ = upper 
quartile; LQ= lower quartile. 


To provide some context for these survey findings, we present some examples from our qualitative 
interview and focus group data. These examples are not meant to definitively explain the variation in 
teachers' reported professional learning hours. They are meant to exemplify the types of school and 
district contexts within which teachers were situated that could be contributing to the variation in 
teachers' professional learning. 

For example, at one high school with high median levels of reported professional learning, but also a 
large interquartile range, teachers who participated in interviews and focus groups reported a number 
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of different opportunities to participate in workshops on topics ranging from behavior management 
program trainings to workshops on "how to work well with ELLs." The principal and instructional coach 
explained that some of these workshops were district led, and others were funded at the school level. 
Teachers also described job-embedded professional learning activities that occurred during weekly staff 
meetings. One focus group teacher characterized the content of these activities as "ideas that I can 
implement in the classroom" immediately, as opposed to new lesson plans to integrate in the future. 
Teachers reported that some professional learning activities were departmentally focused so teachers' 
experiences were likely to vary by department. For example, the participants in a focus group of English 
teachers described professional learning community (PLC) activities that were focused on literacy 
instruction and data use, but teachers in other groups did not explicitly identify PLC activities. 

At another school with high median levels of reported professional learning but a smaller interquartile 
range, teachers who participated in interviews and focus groups reported participating in workshops on 
new curriculum and instructional strategies, as well as data use and data-driven instruction. In contrast 
to the large comprehensive high school discussed in the previous paragraph, this school is a small 
elementary school. The school's faculty was not spread across departments. The principal explained that 
she carefully selected the professional learning activities to ensure that her teachers were all receiving 
training on the programs that she expected them to be implementing in their classrooms and that they 
were not overwhelmed by too many strategies at one time. She also spoke about her regular visits to 
the classroom to ensure that skills taught during professional learning were transferred to daily 
instruction. The majority of interviewed teachers confirmed that they participated in professional 
learning and received classroom visits from the principal. The principal's narrow focus and regular 
follow-up (along with a smaller faculty numbering about 15 teachers) may have helped reduce the level 
of variation in teacher participation within this particular school. 

More teachers reported participating in professional learning on math, literacy, and data use than on 
ELL instruction, special education, or classroom management. In addition to examining the total 
number of professional learning hours, our teacher survey also asked respondents to identify how their 
professional learning hours were spread across six topic areas (see Exhibit 5.2). In analyzing the survey 
data, we separated high school teachers from elementary school teachers because high school teachers 
generally participate in professional learning related to their subject area, whereas elementary school 
teachers are generally multidisciplinary and may participate in professional learning activities spanning a 
number of subject areas. In both high schools and elementary schools, the most common topics were 
math and literacy instructional strategies and data use. 
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Exhibit 5.2. 

Percentage of Teachers Reporting Participating in Professional Learning, 
by Topic, 2011-12 


Topic 

Percentage of Elementary School 
Teachers Reporting Participating in 
Professional Learning 

Percentage of High School Teachers 
Reporting Participating in 
Professional Learning 

Teaching students with disabilities 

31.4% 

43.8% 

Teaching English language learners 

41.3% 

44.2% 

Classroom management 

44.2% 

45.2% 

Mathematics instructional strategies 

61.8% 

78.2% 

Literacy instructional strategies 

77.0% 

84.3% 

Data use 

80.2% 

62.2% 


Source: SST teacher survey, spring 2012. 

Notes: Includes 23 of 25 core sample schools (N = 794 teachers). At the high school level, the percentage of teachers reporting 
participation in "mathematics instructional activities" includes only self-identified mathematics teachers, and the percentage of 
teachers reporting participation in "literacy instructional strategies" includes only self-identified English language arts teachers. 
Two schools were excluded from this analysis because they did not meet the 50 percent response rate threshold on the teacher 
survey. Professional learning hours categorized as "other" were excluded from this analysis. 


Professional Learning That Addresses School Goals and Needs 

Surveys of teachers have identified a number of features of professional learning opportunities 
associated with the likelihood that teachers will report learning new knowledge and skills, and changing 
their practice. These features include how coherent the PD is with a wider set of opportunities for 
learning and development (Garet et al., 2001). One aspect of coherence is the extent to which 
professional learning addresses school needs or goals, which may be associated with an increase in 
overall school capacity (Beaver & Weinbaum, 2012; Elmore, 1997; Garet et al., 2001; Newmann, King, & 
Youngs, 2000). We thus assessed whether respondents perceived their school's professional learning 
opportunities as being tied to school needs. This analysis relies on site visit data from district 
administrators, principals, coaches, and teachers in Year 2 of SIG (2011-12) (see Box 5.2 for more 
information on our analytic approach). 

In six of the core sample schools, respondents felt that their professional learning was aligned with 
school needs and goals. For example, the principal at one elementary school reported that professional 
learning opportunities were developed to respond to perceived deficits in the school's teaching 
capacity. An instructional coach and the principal both noted that prior to the start of the 2011-12 
school year, they examined student data to assess the needs of the students and to identify areas of 
potential weakness. In addition, the coach met with teachers individually to examine each teacher's 
classroom data and to identify where students were struggling. From this work, the coach detected a 
need for students to improve their reading fluency skills and developed professional learning 
opportunities for the teachers to improve their instruction in this area. 

Likewise, the principal at another elementary school described a professional learning plan that was 
created to address both teacher needs and perceived student needs. The teachers participated in 
established PLCs in which they focused on inquiry-based instruction. To continue the work that the 
teachers accomplished in small groups, the principal established a series of peer-to-peer observations. 
Each teacher was observed three times during the year, each time by another teacher or staff member. 
The coach described it as "being paired with peers and going in and having the chance to individually 
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grow with the support of a peer." Teachers described the feedback regarding their strengths and 
weaknesses as helpful in developing their knowledge and skills. 


Box 5.2. Perceptions of Alignment Between Professional Learning 
Opportunities and School Goals and Needs in Core Sample Schools 

School classifications on the alignment between professional learning opportunities and school needs 
are based on Year 2 interview and focus group data (see Exhibit B.10 for more detail on the analytic 
procedures). For this analysis, respondents come from the following respondent groups: district 
administrators, principals, teachers, instructional coaches, and external support providers. 

Aligned 

• At least one respondent in an administrative or support role (e.g., district administrator, 
principal, instructional coach, external support provider) and at least two teachers described 
professional learning opportunities, as a whole, as focused on the goals or needs of the 
school; AND 

• Not more than one respondent explicitly described the professional learning opportunities, as 
a whole, as not focused on the goals or needs of the school. 

Moderately aligned 

• At least one respondent (but fewer than one administrator and two teachers) described the 
professional learning opportunities as focused on the goals and needs of the school, AND no 
respondent explicitly described the professional learning opportunities as not focused on the 
goals or needs of the school; OR 

• At least two respondents described professional learning as a mix of opportunities, some that 
were focused on the goals or needs of the school and some that were not; OR 

• Respondents disagreed about the extent to which professional learning opportunities, as a 
whole, are focused on the goals or needs of the school. 

Minimally or not aligned 

• Not more than one respondent described the professional learning opportunities, on the 
whole, as focused on the goals or needs of the school, AND at least two respondents explicitly 
described the professional learning as not focused on the needs of the school; OR 

• Not more than one respondent described the professional learning opportunities, on the 
whole, as focused on the goals or needs of the school, AND no respondent articulated any 
connections between professional learning opportunities and the goals or needs of the 
school. 


In 15 of the core sample schools, respondents offered mixed perceptions about whether their 
professional learning addressed the needs of the school, and so were classified as moderately aligned. 

Whereas some respondents articulated a clear rationale for specific professional learning opportunities, 
other respondents could not do so, or they expressed the view that the professional learning 
opportunities had an unclear purpose. At the principal and external support provider at one such high 
school identified topic areas for professional learning that included technology use, collaboration, and 
student behavior. Although school leaders described the professional learning opportunities associated 
with these topic areas as "focused" and "consistent," not all teachers expressed these same 
perceptions, nor could they articulate a connection between the opportunities and school needs. 
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Despite being involved with the strategic planning for professional learning at this school, the external 
support provider also perceived some disconnect between professional learning opportunities and the 
school's needs, explaining, "They [school leaders] are just picking the fruit off the tree. They will argue 
that there is a focus. ..but it has to be intense, and we are not doing it in an intense way." 

In the remaining four schools, which were classified as minimally or not aligned, respondents were 
unable to articulate connections between their professional learning and school needs or goals. For 

example, respondents at one high school perceived most of the offerings to be ad hoc. Teachers 
reported that the professional learning opportunities were not tied to performance goals or to 
improvement plans, and the principal identified opportunities for the teachers but did not articulate 
how the opportunities were tied to school goals or student needs. As the instructional coach explained, 
"At the beginning of the year, we sent everyone a list of all the conferences and workshops across the 
state or county, and people had the chance to participate. [The external support provider] put that 
together." The coach did not indicate that the list of professional learning opportunities offered to 
teachers was informed by the principal and instructional coach, or based on student and teacher needs. 

Format of Professional Learning 

Teachers in the core sample schools reported participating in professional learning through multiple 
formats, for example small or large groups and formal or informal settings. Job-embedded professional 
learning is one format that may foster coherence and allow professional learning to be more 
community-centered (Bransford et al., 2000). Indeed, SIG requires that schools provide job-embedded 
professional learning if they are implementing the turnaround or transformation models, defining it as 
follows: 

Professional learning that occurs at a school as educators engage in their daily work activities. It 
is closely connected to what teachers are asked to do in the classroom so that the skills and 
knowledge gained from such learning can be immediately transferred to classroom instructional 
practices. Job-embedded professional development can take many forms, including, but not 
limited to, classroom coaching, structured common planning time, meetings with mentors, 
consultation with outside experts, and observations of classroom practice. (US. Department of 
Education, 2012, p. 30) 

Using this framework and teacher survey data collected in spring 2012, we classified core sample 
schools' reported professional learning into two categories: traditional and job-embedded. Traditional 
formats include, but are not limited to, activities such as workshops, conferences, and college-degree 
courses. These activities tend to be conducted outside the teachers' regular classroom setting, and they 
tend to occur in concentrated blocks of time (e.g., over the summer or on a monthly basis). Our survey 
asked teachers to report the number of hours in which they participated in five different professional 
learning formats: (1) workshops, conferences, institutes, or seminars; (2) course(s) extended over 
several weeks, such as a college course; (3) PLC or meetings of colleagues designed to enhance 
professional learning; (4) formal coaching or mentoring; and (5) other. We categorized the first two 
formats as traditional professional learning and the next two as job-embedded professional learning. 

In 15 of the core sample schools, teachers on average reported spending at least half of their total 
professional learning hours in job-embedded professional learning activities (see Exhibit 5.3). We are 
unaware of any research on striking the optimal balance between job-embedded and traditional 
professional learning formats, but one hypothesis is that it may depend on the context of the school and 
the needs of the teachers. For example, some schools in our sample are in districts that relied quite 
heavily on Teach for America (TFA) to staff their SIG schools. Data from our site visits suggest that TFA 
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teachers in these schools were concurrently working toward their credentials, which may at least 
partially explain why they reported participating in higher levels of traditional professional learning. 
More generally, schools with less experienced teachers might require more traditional coursework and 
workshops to build rudimentary pedagogical and content-relevant skills. 


Exhibit 5.3. 

Percentage of Total Professional Learning Hours Teachers Reported Spending in Job- 
Embedded and Traditional Activities, by Core Sample School, 2011-12 


Blizzard Bay Elementary 
Sawbuck Elementary 
Meribel High 
Peregrine Hill Elementary 
Big Acorn High 
West Marble High 
Gillepsie High 
Coral High 
Melon Elementary 
Rossignol Elementary 
Paul Bunyan High 
Gale Secondary 
Raven Ridge Elementary 
Aerovista High 
Sterling Slope Elementary 
Tyron Elementary 
Sherbrooke Elementary 
Island Bay Elementary 
McAlliston High 
Tyro Trail Elementary 
Baltimore Bridge 
Haven Way Elementary 
Proctor Point High 



■ Job-embedded professional learning DTraditional professional learning 


Source: SST teacher survey, spring 2012. 

Notes: Includes 23 of 25 core sample schools (N = 810 teachers). Two schools were excluded from this analysis because they did 
not meet the 50 percent response rate threshold on the teacher survey. All school names are pseudonyms. 

Job-embedded professional learning includes professional learning communities (PLCs) and formal coaching or mentoring. 
Traditional professional learning includes workshops, conferences, institutes, or seminars as well as courses extended over 
several weeks. Professional learning hours categorized as "other" were excluded from this analysis. 


Using qualitative data from principals, teachers, instructional coaches, and external partners, we sought 
to deepen our understanding of schools' professional learning opportunities. For example, teachers at 
one elementary school reported spending an average of about one third of their PD hours on coaching. 
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The principal, teachers, and coaches explained that their four coaches, as well as the school's 
administrators, provide instruction on teaching practice during weekly common planning sessions for 
each grade and subject. They noted that common planning time allowed teachers who teach the same 
subject areas and grade levels to discuss their plans and align their content. Teachers at this school also 
reported taking part in lesson study, where one teacher would model a lesson and other teachers would 
watch, learn, and share their assessments of the lesson. 

Building School-Level Structures to Foster Learning: 

Teacher Collaboration and Data Use 

For teachers to be able to participate in and benefit from traditional or job-embedded professional 
learning activities, schools must create the necessary conditions and support structures. For example, to 
facilitate the use of professional learning communities (PLCs), schools must set aside regular, structured 
time for teachers to meet and collaborate. Likewise, to facilitate training on the use of data, schools 
(and districts) must promote regular access by teachers to student data. This could involve allowing 
access to district data systems or implementing a unique school-level system for data management and 
acquisition. In this section we examine our core sample schools' efforts to establish support structures 
for teacher collaboration and use of data. 

School-Level Structures to Support Teacher Collaboration 

Some researchers have hypothesized that creating community-centered learning environments that 
encourage teacher collaboration and create communities of practice will enhance teacher learning if 
they include opportunities for shared experiences and discourse around data about student learning 
(Bransford et al ., 2000). Since then, correlational studies have documented a relationship between 
teacher collaboration and indicators of school improvement, including student outcomes (Goddard, 
Goddard, & Tschannen-Moran, 2007; Moller et al., 2013). To create these community-centered learning 
environments, schools could establish structures that facilitate (or in some cases, mandate) 
collaboration. Such structures could include regular meetings for common planning, a cycle of peer 
observation, or release time to jointly develop student assignments or assessments. Schools could also 
develop tools that guide teacher collaboration, such as protocols for PLCs or the expectation that each 
collaborative activity results in a product or tool for shared use (Wenger, McDermott, & Snyder, 2002). 

Drawing on qualitative interview and focus group data, we found that most core sample schools (16 of 
25) reported engaging in efforts to establish conditions or structures to support teacher collaboration 
in Years 1 and 2 of SIG (2010-12) (see Box 5.3). The most prevalent way that schools reported 
supporting teacher collaboration was by establishing a protected time for teachers to work together 
across grades, within a grade, or within a subject. Some schools also established expectations for the 
use of collaborative time. As the principal from one school explained: 

What happened [before] was it was inconsistent. You had a time, you had teachers puiied out; it 
didn't happen all at the same time for the grade levels, so there wasn't a central focus. When 
teachers did try to collaborate, the subs didn't have management in the classroom, so the 
teachers were back and forth. So you could never get the consistency that you needed. So I think 
for us, [we needed] to have some of those structural elements, to have a set time, so now, 

Monday from 3:00 to 4:00 is our professional collaboration time.... SIG has allowed us to do that 
so we could get some of the structures, establishing norms, setting a focus on student learning. 
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Box 5.3. School-Level Structures to Support Teacher Collaboration 

Criteria to determine whether structures were in place to support teacher collaboration are based on 
Year 2 interview and focus group data (see Exhibit B.ll for more detail on the analytic procedures). 
For this analysis, respondents refer to individuals from the following respondent groups: principals, 
teachers, instructional coaches, and external support providers. 

Criteria to determine whether structures were in place to support teacher collaboration are based on 
Year 2 interview and focus group data (see Exhibit B.ll for more detail on the analytic procedures). 
For this analysis, respondents refer to individuals from the following respondent groups: principals, 
teachers, instructional coaches, and external support providers. 

Identified as having structures to support teacher collaboration 

• At least one teacher and respondents in at least one other respondent group specifically 
described one or more of the following: scheduled weekly or monthly time for common 
planning or study groups, guidelines or protocols for efficient use of collaborative time, or 
other opportunities for peer-to-peer discussion of instruction or content; AND 

• Respondents in at least two respondent groups specifically noted that structures to support 
teacher collaboration were new in Year 1 (2010-11) or Year 2 (2011-12) of SIG, or the SIG 
budget provided evidence that SIG supported this structure; AND 

• No respondent explicitly reported that structures to support teacher collaboration had been 
eliminated or reduced in Years 1 or 2 of SIG. 


In contrast, there was no evidence in nine core sample schools of efforts to establish collaborative 
structures during SIG. The principal at one such school explained, "With the cuts, unfortunately 
[common planning] was no longer available." The external provider at another school recognized the 
need for increased collaboration to build capacity but noted the school's current limitations, explaining: 

Then there is the whole issue of building capacity of the adults to meet student needs and how 

we are going organize ourselves to do the work When you start building in collaboration time 

into the school day, that is very expensive time. And that has a lot of implications on your master 
schedule. ..we weren't able to do that [here].... I wasn't able to help them build their master 
schedule in a way that they had time during the day to talk about kids or so that they had time 
together to collaborate by content area. This coming year [Year 3 of SIG], we are going to work 
with the principal on that master schedule and build that time in. 

Among the 8 of these 9 schools that had a sufficient survey response rate, the average percentage of 
teacher survey respondents reporting improvement in collaboration at their school in 2011-12 was 37 
percent. 39 Among the 14 of 16 schools with a sufficient survey response rate that made an effort to 
build collaborative structures, the average percentage of teachers reporting improved collaboration was 
74 percent. Thus, schools that made the effort to establish structures to support teacher collaboration in 


39 The spring 2012 teacher survey included the following item, in which collaboration among teachers was listed as 
one such area: "To what extent has your school improved in any of the following ways during the 2011-12 school 
year?" Response options included "became worse," "stayed about the same," "moderate improvement," and 
"substantial improvement." The latter two response options were counted as having reported improvement in this 
analysis. 
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Years 1 and 2 of SIG were more likely to have teachers who perceived improvements in collaboration in 
Year 2 of SIG. 

School-Level Structures to Support Data Use 

The SIG turnaround and transformation models require schools to "promote the use of data to inform 
and differentiate student instruction" (U.S. Department of Education, 2010, p. 27). In many low- 
performing schools, teachers lack the skills to systematically gather and interpret information on 
student learning and then use it to modify or differentiate their teaching strategies. Teachers may be 
more likely to engage in regular data access and interpretation if districts and schools further establish 
structures that connect teachers to student data. Many districts around the country now facilitate 
teachers' access to data through online data platforms that integrate state assessment results and other 
measures, such as class grades, attendance, and disciplinary actions (U.S. Department of Education, 
2009). Some also have adopted systems of formative or benchmark assessments, often administered 
every six to nine weeks, to provide more frequent and targeted information for teacher decision making 
(Goertz, Olah, & Riggan, 2009; Olson, 2005). In some cases, teachers can select items through an item 
bank, review results in a Web-based platform, and disaggregate data by student subgroup or conceptual 
strand. Schools also may establish other policies or practices that support data use, such as setting aside 
time for data analysis or mandating use of a "data wall." 

Qualitative interview and focus group data show that, overall, most core sample schools (19 of 25) 
reported making an effort to establish structures to support data use, such as workshops on data use 
or hiring data coaches to guide teachers' work in this area, during the first two years of SIG (see Box 
5.4). For instance, an instructional coach at one school explained, "I've never worked for a school so 
data driven and data focused." One of the key structures at this school is the cycle of "data days" at the 
end of every grading period. Students do not attend school on data days, but teachers and 
administrators convene to review testing data and to plan for how to adjust instruction for the next six 
weeks. These days include a sequence of meetings, a structured action plan, and data displays from an 
online system. 


Box 5.4. School-Level Structures to Support Data Use 

Criteria to determine whether structures were in place to support data use are based on Year 2 
interview and focus group data (see Exhibit B.12 for more detail on the analytic procedures). For this 
analysis, respondents refer to individuals from the following respondent groups: principals, teachers, 
instructional coaches, and external support providers. 

Identified as having structures to support data use 

• Respondents in at least two respondent groups specifically described one or more of the 
following: access to diagnostic or benchmark assessments, an online data management 
system and early warning system, or external supports for data analysis; AND 

• Respondents in at least two respondent groups specifically noted that structures to support 
data use were new in Year 1 (2010-11) or Year 2 (2011-12) of SIG, or the SIG budget provided 
evidence that SIG supported this structure; AND 

• No respondent explicitly reported that structures to support data use had been eliminated or 
reduced in Years 1 or 2 of SIG. 


As one teacher from this school explained: 
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At the end of every report period on data day, we have what are called data day conferences 
with a leader of the administration. And so it's one-on-one with me and my principal. I bring my 
reading levels, I bring my math tracker, I bring my benchmark scores, and we sit down and we 
say, "Okay, the kids. ..need to read 75 words per minute, and they're only reading 60. What can I 
do in the next six weeks to get them to read that extra 15 words per minute?" So it's a very 
specific, purposeful 30 minutes of just looking at the data with someone on the leadership team 
who can bring their background experience to you. 

Another teacher from this school added: 

We will start with a whole-staff meeting with the benchmark data from the last six weeks. Later 
in the day we have an action plan, a document we need to fill out, looking at our data with our 
administration and grade team leaders.... We can print out a program called Performance 
Pathways. We can print out many different reports that show the data, each standard, how a 
student performed, what kid missed what question. I can look at it a million different ways, so in 
this Action Plan, I can pinpoint specific kids and skills. 

This example illustrates one school's use of its school calendar as well as a number of data tools (e.g., 
action plan) to facilitate teacher learning around data use. 

In six schools, there was no evidence of efforts to build structures or supports for data analysis in the 
first two years of SIG. Two of these schools had evidence of some infrastructure for data use, but these 
supports were not new in 2009-10 or 2010-11, and appeared to pre-date SIG. In the remaining four 
schools, there was evidence that individual teachers engaged with data, but this engagement was not 
systematic throughout the school. For example, in one such school a teacher explained that she used 
data to guide her own instruction, "...but as a building, we don't view data as analyzing it, looking at it, 
and [revising] what are we going to do. I don't think we're there yet." 

Did Teachers Report Learning from Professional Learning 
Opportunities? 

Using spring 2012 teacher survey data, we examined whether teachers reported learning from their 
participation in professional learning opportunities. Teachers were asked to report if they participated in 
professional learning on seven topics, 40 and if they did participate, they were asked whether they had 
learned from it and changed their teaching, whether they had learned from it but had not changed their 
teaching, or whether they had not learned from it. For core sample high schools, our analysis of 
teachers' responses to professional learning on "mathematics instructional activities" includes only self- 
identified mathematics teachers, and our analysis of teachers' responses to professional learning on 
"literacy/English language arts instructional strategies" includes only self-identified English language arts 
teachers. Two high schools were excluded from this analysis because the total number of mathematics 
and English language arts teachers who responded to the survey was less than five, which we deemed 
insufficient for a meaningful analysis. Another two schools were excluded because of an insufficient 
overall response rate (less than 50 percent). Thus, 21 of 25 core sample schools were included in this 
analysis. 


40 The seven topics were: (1) mathematics instructional strategies, (2) literacy/ELA instructional strategies, (3) 
teaching English language learners (ELLs), (4) teaching students with disabilities, (5) data interpretation and use, (6) 
classroom management, and (7) other activity. 


67 


Case Studies of Schools Receiving School Improvement Grants: Final Report 


In 17 of 21 core sample schools with sufficient data, most teachers reported learning and changing 
their practice after participating in professional learning on math, literacy/English language arts, or 
data use (the three most commonly reported professional learning topic areas). In 12 schools, teachers 
most commonly reported changing their practice in literacy/English language arts (ELA). For example, 90 
percent of teachers at one school reported learning and changing practice in ELA instruction as a result 
of professional learning opportunities, compared with 45 percent for data use and 18 percent for 
mathematics instruction. At another school, 83 percent of teachers reported learning and changing 
practice in ELA instruction as a result of professional learning opportunities, as compared with 38 
percent for data use and 40 percent for mathematics instruction. 

Qualitative data from our core sample schools offer some potential hypotheses about why changes to 
teacher practice were more likely to be reported in some topic areas than others. For example, the 
principal and leadership team at Haven Way reported adopting a fairly narrow focus with respect to 
teacher professional learning. They reported choosing to focus on just a few topics with the expectation 
that doing so would produce significant changes in practice. These areas of focus were closely aligned 
with the principal's perceived school needs. Exhibit 5.4 shows that in the topic areas that the principal 
identified as priorities in 2011-12— ELL instruction, literacy, and data use— teachers reported learning 
and changing practice. 


Exhibit 5.4. 

Percentage of Teachers at Haven Way Elementary Reporting that They Learned and 
Changed Practice as a Result of Professional Learning Opportunities, 2011-12 


Classroom management 
Teaching students with disabilities 
Teaching ELLs 

Data use 

Literacy instructional strategies 
Mathematics instructional strategies 



85% 


0% 20% 40% 60% 80% 100% 


■ Did not participate in activities of this type □ Participated and learned, but did not change teaching 

■ Participated but did not learn very much □ Participated, learned, and changed teaching 


Source: SST teacher survey, spring 2012. 

Notes: Includes 13 of 18 teachers at this school (72 percent survey response rate). 


Likewise, Exhibit 5.4 shows that teachers mostly reported not participating in professional learning 
activities in areas that the principal did not identify as a priority. When asked about the success of the 
professional learning activities in building teacher knowledge and skills and changing practice, the 
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principal explained that her own regular classroom observations confirmed that most teachers had 
changed their practices to incorporate the new strategies being taught. She went on to say that there 
were increases in student test scores for all subgroups of students, including ELLs, which she felt were 
evidence that teachers were using new instructional strategies on a daily basis. 

Chapter Summary 

The total number of hours that teachers reported participating in professional learning opportunities 
during 2011-12 varied across the 25 core sample schools. The total number of professional learning 
hours that teachers reported also varied within schools. Most teachers in most core sample schools 
reported "learning" (gaining new knowledge and skills) from the professional learning opportunities in 
which they participated. Teachers who participated in activities focused on mathematics instruction, ELA 
instruction, and data use were the most likely to report learning and changing their teaching practice as 
a result. The format of professional learning activities also varied. In 15 core sample schools, teachers 
reported spending the majority of their professional learning hours in job-embedded activities. 
Respondents in 6 core sample schools reported that professional learning activities addressed school 
needs, while respondents in 4 schools reported that this was rarely the case. Respondents in the 
remaining 15 schools felt that professional learning activities sometimes addressed school needs. Most 
core sample schools reported making efforts in Year 1 and 2 of SIG to enhance teacher learning by 
establishing policies, structures, and systems that would support teacher collaboration (16 of 25 
schools) or data use (19 of 25 schools). Three schools reported doing neither, although one of these 
schools appeared to already have had an infrastructure to support data use and collaboration prior to 
SIG. 

The results of these analyses are descriptive, and explore schools' reported efforts to provide 
professional learning opportunities. We did not measure all of the features of professional learning, the 
quality of professional learning, teachers' actual participation in these activities, or the extent to which 
teacher learning actually occurred. Thus, we cannot establish with the data we have any relationship 
between professional learning and changes in human capital or student outcomes. However, we can 
establish that most of our core sample schools reported making an effort to build human capital 
through professional learning. That is, most of these schools reported providing teachers with a variety 
of professional learning opportunities, both at the school and through other programs, and supporting 
the ongoing implementation of these skills by creating infrastructure to support their regular use. 
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Chapter 6: District and External Support Providers' 
Efforts to Build Human Capital in SIG Schools 


Research on low-performing schools underscores the role of districts and external support providers in 
the improvement process, explaining that they may provide knowledge, training, and services that 
school-level staff cannot (Finnigan, Bitter, & O'Day, 2009; Flonig, 2004; Massell & Goertz, 2002; O'Day & 
Bitter, 2003; Supovitz, 2006; Zavadsky, 2012). SIG requires districts to ensure that their transformation 
schools receive some form of external support, whether from the state, district, or a designated 
turnaround partner through "ongoing, intensive technical assistance and related support" (U.S. 
Department of Education, 2010, p. 40). SIG also requires districts to "adopt a new governance structure, 
which may include, but is not limited to, requiring the school to report to a new turnaround office in the 
LEA or SEA, hire a turnaround leader who reports directly to the Superintendent or Chief Academic 
Officer, or enter into a multi-year contract with the LEA or SEA to obtain added flexibility in exchange for 
greater accountability" (U.S. Department of Education, 2010, p. 27). With respect to external support 
providers, SIG allows restart schools to contract with education management organizations or charter 
management organizations (EMOs or CMOs), and other SIG schools to partner with "turnaround 
organizations" to help support implementation. 41 

In this chapter, we examine the role of districts and external support providers in supporting efforts to 
build human capital in the 25 core sample schools. With respect to districts, we describe the types of 
supports provided, how districts organized their staff to provide these supports, and the extent to which 
schools found district supports useful. With respect to external support providers, we describe the types 
of organizations that are partnering with our core sample schools to provide supports, explore two 
dimensions of these supports— fit and intensity— and then report the extent to which schools perceived 
partnerships with external support providers to have helped with improvement. 


Box 6.1. Key Chapter 6 Findings 

• Respondents from all 22 schools included in the district support analyses reported that their 
district provided them with at least one of the following: teacher professional learning 
activities (20 schools), principal professional learning activities (15 schools), supportive 
teacher staffing policies (14 schools), and structures and systems to support data use (13 
schools). 

• In 2011-12, 9 of the 13 core sample districts, which served a total of 16 core sample schools, 
reported having sub-districts or designated staff positions in place to support low-performing 
schools. Schools in such districts tended to report receiving support in more areas than 
schools in other districts. 


41 SIG uses the phrase "turnaround organization" as follows: "The LEA might also contract with a turnaround 
organization to assist it in implementing the turnaround model. The LEA might also use external providers to 
provide technical expertise in implementing a variety of components of the school intervention models, such as 
helping a school evaluate its data and determine what changes are needed based on those data; providing job- 
embedded professional learning activities; designing an equitable teacher and principal evaluation system that 
relies on student achievement; and creating safe school environments that meet students' social, emotional, and 
health needs" (U.S. Department of Education, 2010, p. 59). 
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Box 6.1. Key Chapter 6 Findings 

(continued from previous page) 

• Respondents from 10 of the 22 core sample schools included in these analyses perceived their 
district's support as useful to their school's overall improvement efforts. Respondents from 8 
schools expressed mixed perceptions of their district's support, while respondents from 3 
schools indicated that their district's support (or lack thereof) was constraining improvement 
efforts. The 1 remaining school was excluded from this analysis due to insufficient data. In 4 
of the 8 districts with multiple core sample schools, respondents in different schools (within 
the same district) had different perceptions of how useful their district supports were. 

• In 2011-12, 22 of the 25 core sample schools reported receiving support from an external 
support provider to build human capital, such as professional learning (22 schools) or staffing 
support (3 schools). 

• Thirteen schools reported receiving external support from a "SIG partner," an organization or 
individual that was considered central to the change process under SIG. Collectively, these 
schools identified 17 SIG partners, the majority of which reported providing ongoing 
assistance with additional support (11 out of 15 SIG partners with sufficient data) and were 
judged by respondents to be a relevant fit for the school (11 out of 15 SIG partners with 
sufficient data). 

• Overall, respondents described 12 of the 15 SIG partners with sufficient data in positive 
terms, reporting that the services, advice, or feedback they received contributed to the 
school's improvement efforts. Respondents described the remaining 3 SIG partners in mixed 
terms. 


District Efforts to Build Human Capital 

Schools— with the exception of independent charters— exist in the context of district systems whose 
policies and practices may either facilitate or impede the development of human capital. We focus on 
three aspects of district support: the type of supports, how districts organized their staff to provide 
these supports, and the perceived usefulness of the supports. Our findings are based on interview and 
focus group data collected during the 2010-11 and 2011-12 school years in 22 of the 25 core sample 
schools. 42 

Types of District Supports for Building Human Capital 

Our core sample schools reported using staffing and professional learning as two levers to build human 
capital. Although schools sometimes appeared to act on their own— for example, establishing job- 
embedded professional learning communities in their schools— respondents in most of our core sample 
schools reported that their district supported the building of human capital in some way (see Box 6.2). 


42 The three restart schools in the core sample were excluded from these analyses because they are managed by 
EMOs or CMOs. These schools are discussed later in the chapter, when we focus on external support providers. 
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Box 6.2. District Support Services to Build Human Capital in 
Core Sample Schools 

This analysis focuses on four broad categories or domains of district support services— interpreting 
and using data; teacher staffing policies (recruitment and retention); teacher professional learning; 
and building school leadership capacity. Identification criteria for supports are based on Year 2 
interview and focus group data (see Exhibit B.13 for more detail on the analytic procedures). For this 
analysis, respondents refer to individuals from the following respondent groups: principals, district 
administrators, teachers, and instructional coaches. 

Identified as a support 

• The principal and/or a district administrator and a respondent from at least one additional 
respondent group identified one or more specific support service as being supplied by the 
district. For district supports related to teacher staffing and building school leadership 
capacity, the threshold was lowered to reports from the principal and/or a district 
administrator only, as these respondents were in the best position to describe these types of 
supports. 


District and school respondents from 22 schools nested in 13 districts reported that the district 
provided supports for building human capital in at least one of the following four areas: teacher 
professional learning, principal professional learning, teacher staffing policies (recruitment and 
retention), and structures and systems to support data use. Most (14 of 22 schools) reportedly 
received district services in at least three of the four areas. 

Respondents from 20 core sample schools nested in 12 districts reported receiving district supports 
related to teacher professional learning. For example, respondents in one school located in a relatively 
small district reported that the superintendent himself was involved with observing and coaching 
teachers, so that he could better understand how to support them in implementing the pedagogical 
strategies that they were learning about in district-funded workshops. In another set of schools, all in 
the same large urban district, district administrators and teachers described an intensive, weeklong 
"academy" for SIG teachers led by district staff. This academy focused on preparing teachers for the 
school year by reviewing district-sanctioned "models and frameworks" with teachers who were grouped 
by discipline. In contrast, respondents from four schools reported that districts funded teacher 
professional learning but did not take an active role in providing it themselves. Often, district 
administrators and school principals identified a mix of these two approaches where district staff both 
led some of the teacher professional learning and helped school-level staff facilitate other activities on 
their own. 

Principals from 15 schools nested in 8 districts reported receiving district professional learning support 
for school leaders. Examples included one-on-one leadership coaching for principals, cross-school peer- 
group learning networks, and traditional professional learning workshops focused on building leadership 
skills. In one district, all principals who are part of a sub-district supporting low-performing schools are 
reportedly matched with local entrepreneurs. As the district administrator explained, "It's an 
opportunity for principals to be exposed to more entrepreneurial ways of reform, and for the business 
community to see the reality of turning around schools, which allows them to be more supportive." In 
another district, principals reported meeting on a monthly basis to share best practices and problem- 
solve collectively. One principal in this district explained that these collaborative meetings also included 
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visits to each other's campuses to see how best practices are being implemented in an effort to 
"systematize" these best practices across multiple low-performing schools. 

District administrators and principals from 14 schools nested in 8 districts reported district support 
related to teacher staffing. Examples included district policies that offered monetary incentives for 
teachers recruited to teach in hard-to-staff schools, as well as changes to district hiring and firing 
policies that allowed SIG schools flexibility to retain teachers who had less seniority but were considered 
a good fit at the school. One district reportedly keeps all teachers at SIG schools on annual contracts, 
which allows principals to remove ineffective teachers more easily. In another district, a senior 
administrator explained how he has created new leadership positions within SIG schools as a way to 
promote teachers who have completed teaching intern programs (e.g., Teach for America) and give 
them more incentive to stay in the district. 

Lastly, respondents from 13 schools nested in 8 districts reported receiving district supports related to 
creating conditions to support data use. The reported supports included district benchmark 
assessments and district data systems that incorporated tools to allow school leaders and teachers to 
access and manipulate student data. Often, this infrastructure was reportedly accompanied by new 
district data-use policies or district-driven professional learning. At one school, for example, the district 
reported adopting new benchmark assessments and then required all schools to incorporate these 
assessments into their overall approach to data-driven instruction and school improvement. To support 
schools in implementing this new policy, the district reported providing teachers and school leaders with 
ongoing training sessions. 

District Organizational Structures That Support SIG Schools 

The 22 core sample schools included in these analyses are situated in 13 districts. The 13 districts 
reported using two primary organizational structures to support their SIG schools. First, districts created 
sub-districts or offices tasked with providing specialized support to a set of schools with similar needs. 
This approach aligned with SIG, which suggested creating a "turnaround office." Second, districts 
created staff positions responsible for providing specialized support to a set of low-performing schools. 
Sometimes sub-districts or staff members worked solely with SIG schools, and other times schools were 
organized on the basis of performance or geography. 

In 2011-12, 9 of the 13 districts, which served 16 core sample schools, reported having sub-districts or 
staff positions to support low-performing schools (see Box 6.3). Four of these districts, which served 
nine core sample schools, established sub-districts, often referred to as "zones," whose role was 
oversight and support for the turnaround efforts in specific sets of low-performing schools, including SIG 
schools. In one of these districts, SIG reportedly served as the impetus to develop a sub-district to 
provide direct services to the district's SIG schools. In another one of these districts, geographic sub- 
districts existed prior to SIG, but SIG-specific staff were reportedly hired to supplement existing staff in a 
number of these sub-districts. 

Five of these nine districts, which served seven core sample schools, did not create new sub-districts but 
did establish central office positions or teams charged with overseeing and supporting SIG schools. 
Districts that created new staff positions typically reported combining direct service provision with SIG- 
related monitoring and compliance activities. For example, one district used district-level SIG funds to 
hire a full-time staff member tasked with making sure that all of the district's SIG schools were in 
compliance with grant specifications. She was the district's liaison with the state Department of 
Education and monitored grant implementation at each SIG school. However, this staff member also 
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served in a coaching role for SIG principals. Both she and a school principal explained that she was on 
campus weekly and worked with principals to problem-solve issues taking place in their schools. 


Box 6.3. District Organizational Structures to Build Human Capital in Core 
Sample Schools 

This analysis focuses on two specific district organizational structures to support low-performing 
schools, such as SIG schools, in the improvement process: establishment of subdistricts and creation 
of specific staff positions (i.e., central office positions or teams). Identification criteria for structures 
are based on Year 2 interview and focus group data (see Exhibit B.14 for more detail on the analytic 
procedures). For this analysis, respondents refer to individuals from the following respondent groups: 
principals, instructional coaches, teachers, and school improvement team members. 

Identified as a district organizational structure 

• A district administrator and a respondent from at least one additional respondent group 
identified the structure as being in place. 


The remaining four districts, which served six core sample schools, reportedly did not have any 
specialized structures in place to support SIG schools in 2011-12. This does not mean that these 
schools did not receive any district supports, only that SIG schools were not singled out to receive more 
or different supports through new or adapted governance structures designed specifically for that 
purpose. In fact, district administrators and school principals from these six schools explained that the 
district offered uniform supports to all schools, both SIG and non-SIG. 

Although all 22 schools included in this analysis reported receiving at least one type of district support 
for building human capital, the ones that were organized into a sub-district with other SIG or low- 
performing schools or had designated district staff to support SIG schools tended to report receiving 
support in more areas than schools that were in districts not organized in this way. For example, 
respondents in eight schools reported receiving support in all four areas, and seven of these eight 
schools were located in districts with sub-districts or designated staff assigned to support SIG schools. 
The principal at one of these schools explained, "We have probably the most knowledgeable 
administrators in the district [leading our sub-district]. We are given so much support that there is no 
way of not knowing certain things.... Every day we have somebody here [from the district zone]. They 
come and do common planning and train teachers." In contrast, respondents in three of the six schools 
from the remaining four districts with no SIG-specific infrastructure reported receiving support in only 
one area. The principal at one of these schools, where the sub-district structure was dismantled in 
2011-12 because of budget cuts, reported receiving minimal support from the district that year. The 
principal explained, "From the district level, the [sub-district] office was closed, and those district 
support personnel were dispersed throughout other offices.... Our assistant superintendent understands 
the work. ..but we're also 1 of 60 schools now instead of 1 of 6 or 10 schools, so that has some level of 
challenge." 

Perceptions of District Support 

In this section, we examine the extent to which our 22 core sample schools included in the analysis 
above perceived their districts as providing useful supports that facilitated overall school improvement 
(see Box 6.4). Respondents in 10 schools perceived their district's supports as useful, while respondents 
in 8 expressed mixed perceptions. Respondents in three schools indicated that district supports (or lack 
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thereof) were not useful for their improvement efforts. The one remaining school was excluded from 
this analysis of perceptions due to insufficient data. 


Box 6.4. Perceived Usefulness of District Supports to Core Sample Schools 

School-level classifications on perceptions of district support are based on Year 2 interview and focus 
group data (see Exhibit B.15 for more detail on the analytic procedures). Although the data used in 
this analysis primarily focused on the usefulness of district support specifically related to building 
human capital, in some cases, respondents spoke about the usefulness of the district's support as a 
whole, which may have included other types of support. For this analysis, respondents refer to 
individuals from the following respondent groups: principals, other school leaders (assistant 
principals, school leadership team members), teachers, and instructional coaches. 

Perceived district supports as useful 

• The principal and at least one other respondent indicated that district supports were useful to 
school improvement efforts (for district supports related to teacher staffing or building 
leadership capacity, the threshold was lowered to reports from the principal only, as the 
principal was in the best position to describe this type of support); AND 

• The principal described district administrators as accessible and helpful when assisting the 
school to solve problems and overcome challenges to improvement; AND 

• No more than one respondent described district supports as constraining school 
improvement. 

Mixed perceptions of district support 

• The principal disagreed with at least two other respondents about the usefulness of district 
supports; OR 

• The principal and at least one other respondent indicated that certain district supports were 
useful, while others were not useful. 

Perceived district supports as not useful 

• The principal and at least one other respondent indicated that district supports were not 
established, not useful, or were slowing or preventing school improvement efforts; AND 

• The principal described district administrators as inaccessible or difficult to work with when 
assisting the school to solve problems and overcome challenges to improvement; AND 

• No more than one respondent described district supports as useful to school improvement. 


In the 10 schools that perceived their district as providing useful supports, respondents spoke 
positively about their interactions with district staff, the accessibility of district staff, and the overall 
level and types of support they received from the district. For example, district and school respondents 
at one of these schools indicated that a new superintendent and some district office restructuring had 
created an increased level of district support during Year 2 of SIG. Respondents described how the 
district supported the school's use of data-driven inquiry by establishing a new data system, holding 
monthly data collaboration days for teachers, and providing professional learning on data management 
and analysis. The district also reportedly offered a range of teacher professional learning sessions and 
hired two coaches to work with teachers in the school at least one day a week conducting classroom 
observations, lesson debriefs, and lesson demonstrations. Teacher respondents expressed consistently 
positive views of the coaches' work. For instance, one teacher praised her coach for providing regular 
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feedback on her classroom instruction: "She came in once a week, and she left us notes. You know, in 
the past, you might have an administrator come through your classroom three times and then give you 
some feedback. This was feedback every week." The school's principal reported receiving useful 
guidance from district leaders during the school improvement process, and she appreciated how as 
principal of a SIG school, she was afforded special authority in making staffing decisions during a time of 
district layoffs. Rather than being forced to "take the next person off the layoff list," she was able to 
interview candidates from the pool of laid-off teachers and recommend particular individuals for the 
school. 

In another school, district supports were different but also perceived as useful. Here, district 
administrators reportedly took a more responsive approach to service provision. The principal explained 
that the district provided a good base of support, such as professional learning on Common Core State 
Standards and additional curriculum and instruction support for the principal himself. The principal 
explained, "If I'm having an issue or have concerns in various content areas, then I have the freedom to 
call the [district administrator] to say, 'I need help'." However, the majority of the support provided by 
the district was developed by school leaders and submitted to the district for approval, rather than 
provided universally to all SIG schools. The principal explained that his district contacts "completely 
bought into our school improvement initiatives" and gave the school "flexibility to implement its school 
improvement efforts with a few tweakings here and there if necessary." 

In the eight schools with mixed perceptions of the district's support, respondents found some district 
supports useful, but other supports were perceived to be either a poor fit for the school or 
bureaucratic challenges that impeded the school's implementation of improvement strategies. For 

example, when asked whether the district provided guidance on the change process, the principal at 
one school responded, "They do, but they don't." She and an instructional coach noted that the district 
was useful in providing professional learning activities; however, some teacher respondents reported 
receiving unclear and inconsistent information at these district professional learning sessions. 
Respondents at this school also expressed consistently negative views of a district coach assigned to 
serve as a liaison between the school and the central office. The principal described this coach's role as 
duplicative and an inefficient use of the district's limited resources. Members of the school's leadership 
team similarly dismissed the district coach as unhelpful, with one member remarking, "We really don't 
understand why he is here." 

In the remaining three schools where the district was perceived as not useful to school turnaround 
efforts, respondents explained that they received limited guidance and services from district staff, 
found district leaders to be inaccessible, and found the services that they did receive to be either 
insufficient or an impediment to the school's overall improvement process. These three schools were 
in three different districts, two of which had additional SIG schools in our core sample. The other core 
sample schools in those two districts categorized district support as useful or mixed. At the one school in 
the district that did not have additional schools in our core sample, respondents described how the 
school's history of being neglected by the district— its "red-headed stepchild"— continued throughout 
the SIG process. Respondents partly attributed this to the fact that the district central office continued 
to grapple with staff turnover. A school coach explained how this upheaval at the district level 
contributed to the district's lack of awareness and action regarding the school's SIG efforts: 

Because so many changes are going on at the district office, you try to explain the SIG grant, and 
it still doesn't stick because there are so many other things going on right next to them. And 
we're so far [away]; it's not on their radar.... I think the district is the main challenge.... I think we 
might be doomed to repeat history if no one in the district office knows what is going on here. 
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Other respondents explained that the school is in need of more effective staffing policies that would 
enable them to retain the new staff that were hired as a result of the SIG turnaround model. 

In four of eight districts with more than one core sample school, respondents in different schools 
(within the same district) had different perceptions of how useful their district supports were. Possible 
reasons for these differences in perception include differences between schools in the principal's 
familiarity with district staff and district policies, the perceived fit between district supports and the 
school's own needs, and the extent to which schools had flexibility to choose which district services they 
received. 

We focus on two schools from one of our districts to illustrate some of these differences. The principal 
at the first school noted how requests for district coaching support were either rejected or met only 
after substantial "squawking." The principal further explained that the school was hit disproportionately 
hard by district budget cuts, leading him to question whether the district was serious about turning 
around the school: "I do not feel like I have all of the support from the district. There is not the urgency. 

I still believe the SIG grant has been used to balance the budget." The district administrator in the 
turnaround office had a different perspective. When asked about the support that was provided to this 
particular school, the administrator said, "We sat down with [the principal] and said, 'In an ideal world, 
what types of support would you have in place?"' This prompted a discussion of placing district-funded 
coaches at the school who would help teachers with differentiated instruction, specifically for English 
language learners. Another administrator in the same office commented that the district really "believes 
in the principal and his leaders." However, the principal expressed doubt that the district is making 
decisions that are in the students' best interests. 

In contrast, the principal at the second core sample school from this district cited instances in which 
district staff would assist her in "navigating the bureaucracies of SIG," at times jumping in to say, "she's 
got to be able to do that" because of her school's SIG status. The district turnaround officer confirmed 
this, explaining that a major component of his job was to "be a facilitator or a blocker to shield that 
school from some of the noise that can surround it." 

One major difference between the two school principals from this district was their familiarity with the 
district. The principal from the second school had been a long-time district administrator who was well 
connected and who understood how district policies would influence her school and whom she would 
need to speak with if she needed to resolve problems or access services. The principal from the first 
school, while a long-time educator, was a newer principal (three years of experience) and had never 
worked at the district level. Although we cannot definitively determine whether the second principal 
was better able to protect her school from such changes due to her "insider" status in the district, it 
appears that having strong connections to district administrators may have helped her negotiate 
supports for her school more effectively than the first principal. 

In another district, the principal at one of the core sample schools described certain district supports as 
useful but the majority of them as "a waste and part of a district agenda that I do not support." In 
contrast, the principal at another school in that same district described these same supports as "very 
meaningful" and explained, "I have been very supported by the district. I've never seen it like this in all 
my years as an administrator." The district administrator did not provide evidence to support why one 
school would perceive district supports as more useful than another within the sub-district, and both 
schools were supported by the same district personnel. However, low-performing schools in this district 
are reportedly grouped on the basis of their performance, with higher-performing schools given more 
flexibility to participate in district-provided professional learning depending on the principal's 
perceptions of what the school needs. The second school was part of this higher tier of low-performing 
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schools, and so this potentially could have influenced the extent to which school respondents perceived 
district supports to be more useful than those at the first school. 

External Support Provider Efforts to Build Human Capital 

Districts often partner with external support providers to deliver services to low-performing schools 
when they are not able to offer these services themselves. Likewise, schools often seek out external 
partners when leaders identify a need that the school cannot address. SIG encourages its grantees to 
partner with external support providers— such as school turnaround organizations, education 
management organizations (EMOs), and charter management organizations (CMOs)— to work with 
schools to facilitate the turnaround process (U.S. Department of Education, 2010, p. 40). Depending on 
local charter authorization laws, CMOs or EMOs can take on different roles. Some assume all traditional 
responsibilities of a school district, and others are limited to professional learning, instructional 
strategies, and curricular support. The three restart schools in our core sample reported receiving a 
range of supports, each with a different relationship with the district. Because of this variation, we did 
not include CMOs and EMOs in the analysis of district supports summarized in the prior section. In the 
analyses that follow, we include these restart schools, bringing our sample to 25 schools. 

In this section, we describe external support provider efforts to help our core sample schools build the 
capacity of teachers and school leaders. First, we describe the broad set of external individuals and 
organizations to which our schools turned for support, as well as the areas in which these organizations 
supported the schools. Next, we focus in on 13 of the 25 core sample schools where principals identified 
an organization that they considered to be their primary external support provider for SIG. In the 
analyses of these SIG partners, we explore two dimensions of external support that have been posited 
to be associated with quality (Boyle et al., 2009): intensity and fit. 43 Last, we describe our schools' 
perceptions of the usefulness of support provided by the SIG partners. 

Number and Types of External Support Providers 

In 2011-12, 22 of the 25 core sample schools reported receiving supports to build human capital from 
organizations other than the district. Collectively, these schools identified 53 different external 
providers, who provided supports related to building human capital. Among these 22 schools, 19 
reported receiving support from 1 to 3 providers, while the remaining 3 schools reported receiving 
support from 4 to 6 providers. Providers included colleges and universities, CMOs, EMOs, school 
turnaround organizations, curriculum developers, consultants from SEAs, and other organizations (such 
as local counties or advisory boards). 

It is unclear what the optimal number of support providers for any given low-performing school is 
because low-performing schools have different performance problems, and support providers engage 
with schools in varied ways. Flowever, previous qualitative studies of external support have highlighted 
cases in which multiple providers were perceived to send mixed messages and limit the coherence of 
the support (Boyle et al., 2009). Among the core sample schools, there is evidence that teachers in the 
two schools with the highest number of support providers (five and six providers) were also more likely 
to strongly agree with the survey statement: "I worry that we are adopting too many different programs 
and practices in this school." Although we cannot infer any causal relationship between a high number 


43 Other hypothesized dimensions of higher-quality support include coherence, timeliness, responsiveness, and 
stability, but our data were insufficiently detailed to explore these dimensions. 
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of support providers and these survey responses, it is possible that multiple support providers could 
introduce many different programs and practices that might detract from programmatic coherence. 

Areas of External Support 

Respondents from all 22 of the core sample schools that reported working with an external support 
provider indicated that the provider delivered some form of professional learning (See Box 6.5). For 
example, respondents from 20 schools reported receiving professional learning on curriculum and 
instruction, and respondents from 8 schools reported receiving professional learning on leadership 
development. Data use and culture and climate were other reported professional learning topics. At one 
of these schools, an external provider reportedly offered training to all teachers in the school regarding 
strategies for building personal connections with students and fostering a civil learning environment in 
each classroom. The format of the professional learning varied. In one school, the school turnaround 
organization delivered a three-day teacher workshop at the beginning of the school year on how to 
create, administer, and use common formative assessments as part of instruction. In another school, the 
curriculum developer delivered math-related professional learning to teachers through classroom 
observations and debriefing. Based on those observations, the developer then facilitated a one-day 
workshop that focused on analyzing classroom lessons. 


Box 6.5. Areas of External Support Provider Support to Build Human Capital 

This analysis focuses on 10 areas of external support to improve the capacity of school leaders and 
staff, identified through the Year 2 qualitative data. These areas include professional learning related 
to (1) budget planning, (2) coaching, (3) community involvement, (4) curriculum and instruction, (5) 
data use, (6) school culture and climate, (7) school leadership, (8) staff collaboration, and (9) strategic 
planning as well as staffing support related to (10) teacher recruitment, retention, or evaluation. 
Identification criteria for structures are based on Year 2 interview and focus group data (see Exhibit 
B.16 for more detail on the analytic procedures). For this analysis, respondents refer to individuals 
from the following respondent groups: district administrators, principals, teachers, school 
improvement team members, and instructional coaches. 

Identified as an area of support 

• The principal or a school improvement team member, and a respondent from at least one 
additional respondent group, identified the support as being supplied by the external support 
provider. 


Respondents from 3 of the 22 core sample schools that reported working with an external support 
provider indicated that the provider gave staffing support by facilitating efforts to recruit, retain, or 
evaluate staff. These three schools were implementing the restart model, in which an external 
organization assumes management responsibility for schools. For example, the CMO's Fluman Resources 
Department reportedly supported the teacher-hiring process at one of these schools by pooling 
applications and recruiting teachers. At another one of these schools, the EMO reportedly developed a 
tool based on a teaching and learning framework that the principal found helpful in evaluating and 
giving feedback to teachers. The principal explained, “Now when I come into any teacher's classroom, I 
pull out my [tablet], and I'm on there, and I'm scripting what the teacher is doing and the students are 
doing.... [The teachers] know what to expect, and if I walk out of that room, and they go to their 
computer, and they don't see my e-mail, they will come down and say 'I didn't get my feedback'." 
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Support From Primary SIG Partners 

We now turn to a subset of 13 core sample schools, whose respondents identified a specific SIG 
partner, which is an organization or individual that the school leaders considered central to the 
change process under SIG. Collectively, these 13 schools identified 17 SIG partners, of which 3 were 
CMOs and EMOs for the 3 restart schools in our sample while the remaining were partners for the other 
10 schools and were not involved with the direct oversight and management of the school. Of the 17 SIG 
partners, 16 reported working with teachers, 14 reported working with principals, and 8 reported 
working with school improvement teams and instructional coaches. Most partners reported working 
with multiple groups within each school. For example, at one school, the SIG partner reportedly 
provided a broad range of curriculum and instructional supports, and in doing so interacted with the 
principal, curriculum coordinator, teachers, and school leadership team. We examined two dimensions 
of quality associated with the support provided by these SIG partners (see Boxes 6.6 and 6.7): 

• The intensity of support— in terms of the number of days of support or the span of time over 
which support is provided— may influence the likelihood of fostering and sustaining school-level 
change (Davis, McDonald, & Lyons, 1997; Education Alliance at Brown University, 2006; 

Finnigan, Bitter, & O'Day, 2009; Laguarda, 2003; Mintrop & Trujillo, 2005; Reville, 2007). 

• The fit of support encompasses many features, including the alignment of the expertise of a 
support provider to a specific school's needs, and the fit between a school's challenges and the 
selected intervention (David, Kannapel, & McDiarmid, 2000; Rennie Center, 2005). If there is a 
mismatch— for example, a school improvement facilitator with suburban experience is assigned 
to a rural school— it may be more difficult to foster meaningful dialogue, to identify appropriate 
interventions, to sustain improvement actions, or to garner the respect and trust of school 
personnel. 


Box 6.6. Intensity of SIG Partner Supports 

School-level classifications on the intensity of supports provided by SIG partners (e.g., external 
support providers that school leaders considered central to the change process under SIG) and core 
sample schools are based on Year 2 interview and focus group data (see Exhibit B.17 for more detail 
on the analytic procedures). For this analysis, external support provider refers to the external support 
provider who served as the school's SIG partner, and respondents refer to individuals from the 
following respondent groups: principals and external support providers. 

Concentrated 

• The principal or external support provider reported that SIG partner provided supports 
through one-time professional learning activities. 

Ongoing 

• The principal or external support provider reported that SIG partner provided supports 
regularly throughout the year. 

Ongoing with additional supports 

• The principal or external support provider reported that SIG partner provided supports that 
are ongoing with additional supports (i.e., one-time professional learning activities; work in 
summer; remote supports). 
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Box 6.7. Perceived Fit Between SIG Partners and Core Sample Schools 

School-level classifications on the fit between SIG partners (e.g., external support providers that 
school leaders considered central to the change process under SIG) and core sample schools are 
based on Year 2 interview and focus group data (see Exhibit B.18 for more detail on the analytic 
procures). For this analysis, external support provider refers to the external support provider who 
served as the school's SIG partner, and respondents refer to individuals from the following respondent 
groups: principals, external support providers, and school improvement team members. 

Relevant 

• The external support provider and either the principal or a member of the school 
improvement team described the external support provider as having relevant experience, 
defined as expertise working with the particular school or schools with similar characteristics, 
experience working with teachers or administrators, and prior teaching/administrative 
experience; AND 

• No respondent mentioned any specific shortcomings of the external support provider's 
experience and expertise. 

Moderate 

• The external support provider and either the principal or a member of the school 
improvement team described the external support provider as having some relevant 
experience, but mentioned shortcomings in one of the three areas: expertise working with 
the particular school or schools with similar characteristics, experience working with teachers 
or administrators, and prior teaching/administrative experience. 

Low 

• The external support provider and either the principal or a member of the school 
improvement team described the external support provider as not having relevant 
experience, mentioning shortcomings in at least two of the three areas: expertise working 
with the particular school or schools with similar characteristics, experience working with 
teachers or administrators, and prior teaching/administrative experience. 


The majority of SIG partners (11 of 15 with sufficient data) reported providing ongoing assistance with 
additional support to the core sample school that they served. These SIG partners reported 
supplementing their regular, on-site assistance with additional professional learning activities, summer 
activities, or electronic communication. For example, the SIG partner at one school reported delivering 
monthly professional learning support to the principal, in addition to weekly visits with other staff. To 
get ready for the school year, the SIG partner also reported delivering a weeklong professional learning 
session to teachers and a separate weeklong session to the principal during the summer. 

The remaining four SIG partners who provided sufficient data on the frequency and duration of their 
assistance reported providing ongoing support to their core sample school. Thus, all 15 SIG partners 
included in this analysis reportedly worked with their core sample school on a regular basis throughout 
the school year. Flowever, the frequency of interactions among SIG partners and the schools they served 
varied. Twelve SIG partners were reportedly present in the schools on a weekly basis throughout the 
school year. For example, the turnaround officer at one school was reportedly on-site two days each 
week, meeting with the leadership team, visiting classrooms, supporting instructional coaches, and 
spending time with the principal in his office or during lunch duty. Because the school is small, the SIG 
partner concluded that "two days a week is plenty" to deliver support. In another school, the SIG 
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partner reportedly provided weekly support for a few months but then tapered off. The remaining three 
SIG partners reported providing support every other week, monthly, or every other month. 

The majority of SIG partners (11 of 15 with sufficient data) were judged by respondents to be a 
relevant fit for the school, in terms of the partner's expertise and experience. For example, the SIG 
partner at one school was perceived to be a good match because he had a background as a teacher and 
principal in urban, low-performing secondary schools. Likewise, the SIG partner at another school had 
experience as an administrator of a low-performing rural school, but his background in higher education 
reportedly provided a broader perspective on the school improvement process. As the principal 
explained: 

I actually sat on the interview panel when we were going through to hire for that position, and 
from the perspective of "This is the person I'm going to work with," I sat and I listened to him, 
and his background is rather interesting. He's been a high school principal, I want to say, at four 
different sites. He was a college professor and then an assistant superintendent for the county. 

So his experience was pretty vast. 

The principal also explained that an external support provider must "develop a really professional solid 
working relationship with the site administrator.... It's a relationship of trust that seems to work out 
pretty well for us at least." 

The other 4 of 15 SIG partners with sufficient data were judged to be a moderate fit for the school. For 

example, both individuals from one of these SIG partners, who were working with the principal 
extensively on strategic planning, had worked 10 years in education and had some limited experience 
with school leaders. Flowever, these partners lacked personal experience working in urban or low- 
performing schools or as school administrators in any context. The individual from another one of these 
SIG partners had a background at the elementary level and stated that it was a shift for her to come to 
the high school. She discussed her challenge of working in a high school given her background and trying 
to make sense of the organization as a whole. She explained, "What is different is the sheer magnitude. 
When you think about how decisions are made and how things are communicated— who is moving all 
the pieces? It's harder for me to figure out that system from a birds-eye view because I have less 
experience being in it." 

Perceptions of SIG Partners' Support 

Overall, respondents described 12 of the 15 SIG partners with sufficient data in positive terms, 
reporting that the services, advice, or feedback they received contributed to the school's 
improvement efforts (see Box 6.8). A district administrator for a school served by 1 of these 12 partners 
attributed gains in student scores to the professional learning activities that the partner provided, and 
teachers described their SIG partner positively by saying that feedback was "really helpful," "useful," 
and provided "right away." One teacher explained: 

I like the input from the [SIG partner]. They come in. ..they observe you in the classroom, so they 
give you like a critigue, and it's really good because sometimes you don't realize. ..that you could 
be doing something better. Or they give you ideas to make your teaching easier and more 
beneficial for all the learners. 

Similarly, the instructional coaches at one school described one of their SIG partner's support to the 
leadership as "wildly helpful" and "very helpful." The SIG partner reportedly helped to create end-of- 
year goals for the coaching staff related to the theory of action. According to one coach, "It was great to 
have someone facilitate who was not part of the group. [The SIG partner] helped us gel as coaches 
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together." The principal also reflected on the usefulness of the external support by saying that the SIG 
partner helped the coaches to look at their coaching model and apply best practices. In describing the 
result, she said, "It is really clear who is getting coached. It is very transparent; everyone knows they're 
going to get coaching at some point this year." The principal, coaches, and school improvement team at 
another school described the SIG partner as an "invaluable resource" in terms of "vision and planning, 
aligning professional development, building professional learning communities." One school 
improvement team member explained that the SIG partner "can see what you can't see. So that is one 
of the values of [the SIG partner]." 


Box 6.8. Perceived Usefulness of SIG Partner Support 

School-level classifications on the extent to which respondents perceived the supports provided by 
SIG partners as useful are based on Year 2 interview and focus group data (see Exhibit B.19 for more 
detail on the analytic procedures). Although the data used in this analysis primarily focused on the 
usefulness of SIG partner support specifically related to building human capital, in some cases, 
respondents spoke about the usefulness of the SIG partner's support as a whole, which may have 
included other types of support. For this analysis, respondents refer to individuals from the following 
respondent groups: district administrators, principals, teachers, instructional coaches, and external 
support providers. 

Perceived SIG partner supports as useful 

• At least two respondents described the SIG partner in predominately positive terms; AND 

• No more than one respondent holds a negative perception of the SIG partner. 

Mixed perceptions of SIG partner supports 

• Respondents disagreed with one another about the usefulness of supports provided by the 
SIG partner; OR 

• Respondents indicated that certain supports provided by the SIG partner were useful, while 
others were not useful. 

Perceived SIG partner supports as not useful 

• At least two respondents described the SIG partner in predominantly negative terms; AND 

• No more than one respondent holds a positive perception of the SIG partner. 


Respondents described the remaining 3 of 15 SIG partners in mixed terms. For example, although the 
principal at one school asserted that their EMO helped the school in developing and implementing a 
teacher evaluation tool, they lacked knowledge about budget planning and human resources. One 
teacher perceived that the EMO provided some "excellent" training activities, but the instructional 
coach explained, "I don't know that their focus is [for] improving instruction. I know they can identify 
what's bad and what's good, but I don't know [that] they can identify why it's bad, why it's good, and 
how they're going to impact what they're seeing. It's just like drive-by." 

Chapter Summary 

This chapter described the types of supports related to human capital that core sample schools received 
from their district and external support providers, and how useful the schools found these supports in 
facilitating their improvement efforts. Respondents from all 22 schools and 13 districts included in this 
analysis reported that their district provided some form of assistance, such as professional learning 
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activities for teachers (20 schools) or principals (15 schools), supportive staffing policies (14 schools), or 
data use structures (13 schools) to help build school capacity. Some larger, urban districts chose to 
reorganize their SIG and low-performing schools into sub-districts, and other districts chose to assign 
specific central office staff to support SIG implementation and school turnaround more generally. In 
both types of districts, school leaders described receiving more supports than in those districts without 
such organizational structures in place. However, this finding does not necessarily mean that such 
organizational structures actually led to better district support since many contextual factors, including 
district size, location, and the historical relationship that existed between district administrators and 
school leaders, may also be relevant. Overall, respondents from 10 of the 21 core sample schools with 
sufficient data indicated that the support they received from the district was useful, but respondents in 
8 schools described mixed perceptions of their district's support, and respondents from 3 schools 
reported that the district's support (or lack thereof) was constraining their efforts to improve school 
capacity. 

Similarly, all 25 core sample schools reported receiving support from at least 1 external provider. The 
types of support varied but were most often related to professional learning (25 schools) or curriculum 
and instruction (20 schools). Across the 25 schools, respondents identified 53 external partners that 
supported the development of human capital, and respondents in 13 of the 25 schools identified SIG 
partners that were taking a more significant role in supporting the school improvement process. In this 
subset of schools, external providers generally provided intense, ongoing support, and appeared to be at 
least moderately experienced in the areas in which they were providing support. Although we cannot 
determine whether these SIG partners succeeded in building human capital in these schools, school 
respondents described the majority of these partners (12 of the 15 SIG partners with sufficient data) as 
valuable resources offering useful insights and assistance in the school change process. 
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Chapter 7: Conclusion to Part II 


One possible reason persistently low-performing schools remain low-performing for a number of years 
is that they lack the necessary capacity to improve on their own. Consistent with this hypothesis, the SIG 
intervention models specify improvement actions intended to help build human capital in these 
schools— through hiring new staff, providing professional learning opportunities, and providing external 
support (U.S. Department of Education, 2012). In Part II of this report, we have described the extent to 
which our 25 core sample SIG schools appeared to put forth effort to build human capital during the first 
two years of SIG. The human-capital-building activities we individually analyzed were: 

• Chapter 3: Principal replacement, distributed leadership 

• Chapter 4: Teacher replacement, adding new noninstructional staff positions 

• Chapter 5: Professional learning opportunities, structures, and supports for data use and 
collaboration 

• Chapter 6: District support, external support providers 

We conclude this part of the report by synthesizing the various efforts that schools reported making in 
the areas above and consider whether some schools put forth more or less overall effort in their human- 
capital-building activities. We first look at our quantitative measure that aggregates the efforts of each 
school in the activities above. We then present narrative case studies of two core sample schools to 
provide a qualitative perspective on building human capital overall. We conclude with a brief summary 
of Part II findings. 


Box 7.1. Key Chapter 7 Findings 

• In addition to variation in the specific activities that each school implemented (as reported on 
in Chapters 3 to 6), our aggregate measure suggests variation across schools in the number of 
efforts to build human capital during the first two years of SIG. Aggregate ratings of school 
efforts to build human capital ranged from 2.5 to 8.5 out of a maximum possible rating of 10. 

• The most common approach to building human capital among core sample schools was 
adding noninstructional staff positions. All but one core sample school reported adding such 
staff in either Year 1 or 2 of SIG, and 13 of these schools reported adding such staff in both 
years. 

• The four schools in which respondents described SIG as the primary impetus for change in 
Year 1 of SIG (see Le Floch et al ., 2014) had a higher aggregate score of human-capital- 
building efforts than did schools in which SIG was not perceived to be an impetus for change. 


Efforts to Build Human Capital: An Aggregate View 

Based on the findings reported in Chapters 3 to 6, all 25 core sample schools reportedly engaged in at 
least some efforts to build human capital during the first two years of SIG. They replaced teachers to 
varying degrees, added noninstructional staff, and in some cases, moved staff within the school to 
better meet students' needs. Many teachers in these schools reported engaging in a high volume of 
professional learning opportunities, and in at least some schools, these opportunities appeared to be 
aligned with school needs and goals. Some schools also reported developing policies and structures that 
supported data use, collaboration, and distributed leadership. Finally, districts identified individuals or 
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established sub-districts through which SIG schools could receive targeted support— although some 
schools perceived their district context to be constraining rather than supportive. 

To better understand each school's overall efforts and the differences among them, we developed an 
aggregate measure that combined the individual results of selected analyses presented in Chapters 3 to 
6 (see Box 7.2). Briefly, we credited schools for efforts to build human capital if the findings presented in 
Chapters 3 to 6 showed schools or districts engaging in specific human-capital-building activities. 

Specific indicators include: 

• Efforts to build structures for distributed leadership (see Chapter 3 and Exhibit B.3) 

• Replacement of 50 percent of teachers (see Chapter 4 and Exhibit B.4) 

• Creation of nonteaching staff positions (see Chapter 4 and Exhibit B.6) 

• Purposeful approach to staffing (see Chapter 4 and Exhibit B.7) 

• Alignment of professional learning with school goals and needs (see Chapter 5 and Exhibit B.10) 

• Hours of teacher-reported professional learning (see Chapter 5) 

• Efforts to build structures to support teacher collaboration (see Chapter 5 and Exhibit B.ll) 

• Efforts to build structures to support data use (see Chapter 5 and Exhibit B.12) 

• Presence of district organizational structures (e.g., sub-districts or designated staff to support 
SIG schools) (see Chapter 6 and Exhibit B.14) 

• Presence of SIG partner to support school (see Chapter 6 and Exhibit B.17) 


Box 7.2. Aggregate Measure of Efforts to Build Human Capital 

For each capacity-building indicator, analysts ascribed numeric values to the classifications, which 
were summed to create an aggregate index of capacity-building efforts, with a maximum possible 
score of 10 (see Exhibit B.20 for more detail on the analytic procedures). Most indicators were scored 
according to a binary system, with 0 assigned to the negative category (e.g., "no evidence," "not 
identified") and 1 to the affirmative category (e.g., "identified"). For example, schools rated as having 
a "purposeful approach to staffing decisions" were assigned a 1, and schools rated as having "no 
evidence of a purposeful approach to staffing decisions" were assigned a 0. The only indicators that 
did not follow this binary system were: 

• Mean hours of teacher-reported professional learning. Although scored according to a binary 
system, 0 represents below 111 hours and 1, at or above 111 hours [Based on the National 
Longitudinal Study of NCLB, the national estimate of the average number of PD hours for 
teachers in Title I schools identified for improvement was 111 hours (Taylor et al ., 2010)]. 

• Creation of nonteaching staff positions. This indicator was scored using a three-point scale 
with 0 assigned to schools that added non-instructional staff positions in Year 1 only, 0.5 
assigned to schools that added non-instructional staff positions in Year 2 only, and 1 assigned 
to schools that added non-instructional staff in both years. 

• Alignment of professional learning with school goals and needs. This indicator was scored 
using a three-point scale with 0 assigned to schools rated as minimally or not aligned, 0.5 
assigned to schools rated as moderately aligned, and 1 assigned to schools rated as aligned. 


Note that this measure reflects an effort to build capacity, not whether a school actually improved 
capacity. For example, with regard to the number of professional learning hours, the most we can 
conclude is that schools in which teachers reported receiving a large number of professional learning 
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hours exerted some effort to build teachers' knowledge and skills, but we do not know whether the 
professional learning was of high quality or if it actually enhanced human capital. Also note that this 
measure may oversimplify and obscure important aspects of the capacity-building process in some 
cases. For example, school leaders might reasonably opt to focus on a few leverage points rather than 
across-the-board efforts. Nevertheless, since SIG prescribes numerous activities for its grantees under 
each model, we believe that this aggregate perspective provides information on each school's holistic 
efforts to build human capital that would be lost if we were only to consider each activity in isolation. 

In addition to variation in the specific activities that each school implemented (as reported on in 
Chapters 3 to 6), our aggregate measure suggests variation across schools in the number of efforts to 
build human capital during the first two years of SIG (see Exhibit 7.1). Overall, the aggregate ratings of 
school efforts to build human capital ranged from 2.5 to 8.5 out of a maximum possible rating of 10, 
with 12 schools scoring above 6, and 13 scoring 6 or below. There was no apparent association between 
a school's aggregate measure of efforts to build human capital and its SIG intervention model, external 
context, urbanicity, or teachers' average years of experience. 

The most common approach to building human capital was adding noninstructional staff positions 

(see Exhibit 7.1). All but one core sample school reported adding staff in either Year 1 or 2 of SIG, and of 
these schools, 13 reported adding staff in both years. In addition, 19 of the 25 core sample schools 
developed structures and policies to support data use for instructional decisions, an approach consistent 
with the SIG transformation and turnaround models. Relatively fewer schools (13) identified an external 
organization or individual they had hired specifically to support them throughout the turnaround 
process, despite SIG's emphasis on seeking support from external partners. 

The four schools in which SIG was described as an impetus for change in Year 1 of SIG (see Le Floch et 
al., 2014) had somewhat higher average ratings than schools in which SIG was not an impetus for 
change (7.75 vs. 5.67). To better understand how multiple human-capital-building efforts fit together in 
each school and how they can be influenced by the specific context within which each school is situated, 
we have included case study profiles for 2 of the 25 core sample schools (see Box 7.3 and Box 7.4), 
selected because they were situated in different contexts and fell on different ends of our aggregate 
scale. 
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Exhibit 7.1. 

Aggregate Measure of School Efforts to Build Human Capital, 2010-11 and 2011-12 



Efforts to 

Build 

Structures for 

Collaboration 

Efforts to 

Build 

Structures for 

Data Use 

Efforts to Build 

Structures for 

Distributed 

Leadership 

Replaced 

Teachers 

Added Non- 

instructional 

Staff 

Positions 

Purposeful 
Approach to 
Staffing 

Purposeful Approach 

to Professional 

Learning 

Mean Hours 

of 

Professional 

Learning 

District 

"Zone" or 

Staff to 

Support SIG 

SIG Partner 

Overall 

Aggregate 

Score 

Paul Bunyan High 

no 

no 

no 

no 

Yr 1 

no evidence 

minimally purposeful 

36 

yes 

SIG partner 

2.5 

West Marble High 

no 

no 

no 

yes 

Both 

no evidence 

minimally purposeful 

112 

no 

none 

3 

Raven Ridge Elementary 

no 

yes 

no 

no 

Yr 2 

no evidence 

minimally purposeful 

136 

no 

SIG partner 

3.5 

Melon Elementary 

no 

yes 

no 

no 

Yr 2 

no evidence 

moderately purposeful 

99 

yes 

none 

4 

Coral High 

yes 

yes 

yes 

no 

no 

no evidence 

moderately purposeful 

100 

yes 

none 

4.5 

Sherbrooke Elementary 

no 

yes 

no 

no 

Both 

no evidence 

moderately purposeful 

120 

no 

SIG partner 

4.5 

Peregrine Elementary 

no 

yes 

yes 

yes 

Yr 1 

no evidence 

moderately purposeful 

96 

yes 

none 

5 

Proctor Point High 

yes 

yes 

no 

no 

Yr 1 

purposeful 

moderately purposeful 

224 

no 

none 

5 

Elmsville High 

no 

no 

yes 

no 

Both 

no evidence 

highly purposeful 

134 

yes 

none 

5 

Tyro Trail Elementary 

no 

yes 

no 

yes 

Yr 1 

purposeful 

minimally purposeful 

127 

yes 

none 

5.5 

Sawbuck Elementary 

yes 

yes 

no 

yes 

Both 

no evidence 

moderately purposeful 

105 

no 

SIG partner 

5.5 

Aerovista High 

yes 

yes 

no 

no 

Yr 1 

purposeful 

moderately purposeful 

117 

yes 

none 

6 

Rossignol Elementary 

yes 

no 

yes 

no 

Both 

purposeful 

highly purposeful 

143 

no 

none 

6 

Tyron Elementary 

yes 

no 

yes 

yes 

Both 

no evidence 

moderately purposeful 

108 

yes 

SIG partner 

6.5 

Inner Brooks High 

yes 

yes 

yes 

yes 

Yr 2 

no evidence 

highly purposeful 

126 

no 

none 

6.5 

McAlliston High 

yes 

no 

yes 

no 

Both 

no evidence 

moderately purposeful 

169 

yes 

SIG partner 

6.5 

Haven Way Elementary 

yes 

yes 

yes 

no 

Yr 1 

purposeful 

moderately purposeful 

178 

yes 

none 

7 

Gale Secondary 

yes 

yes 

yes 

no 

Both 

purposeful 

moderately purposeful 

70 

yes 

SIG partner 

7.5 

Blizzard Bay Elementary 

yes 

yes 

yes 

no 

Yr 2 

purposeful 

highly purposeful 

91 

yes 

SIG partner 

7.5 

Big Acorn High 

no 

yes 

yes 

yes 

Both 

purposeful 

moderately purposeful 

97 

yes 

SIG partner 

7.5 

Sterling Slope Elementary 

yes 

yes 

yes 

yes 

Yr 2 

purposeful 

moderately purposeful 

192 

yes 

none 

8 

Baltimore Bridge 
Elementary 

yes 

yes 

no 

yes 

Both 

purposeful 

highly purposeful 

130 

no 

SIG partner 

8 

Gillepsie High 

yes 

yes 

yes 

no 

Both 

no evidence 

highly purposeful 

155 

yes 

SIG partner 

8 

Island Bay Elementary 

yes 

yes 

yes 

yes 

Both 

no evidence 

moderately purposeful 

201 

yes 

SIG partner 

8.5 

Meribel High 

yes 

yes 

yes 

yes 

Both 

purposeful 

moderately purposeful 

183 

no 

SIG partner 

8.5 


Source: SST respondent interviews and focus groups, spring 2011 and spring 2012; SST teacher survey, spring 2011 and spring 2012. 
Notes: Includes 25 core sample schools. All school names are pseudonyms. 
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Box 7.3. Gale Secondary's Efforts to Build Human Capital 

Gale Secondary is located in a rural community about 15 miles from a somewhat larger town in a 
relatively isolated part of the state. The area, as described by the principal, is economically depressed 
and was hit extremely hard during the recession because its agricultural and industrial base "dried up 
and disappeared." The nearby downtown is small and rather quiet. There are a few areas of activity, 
but many storefronts are either run-down or closed. Coinciding with the recession, focus group 
respondents frequently voiced concern about unemployment, which rose to double-digit levels during 
SIG. 

The principal in Year 1 of SIG (2010-11) was perceived to be a weak instructional leader with limited 
communication skills. With regard to his vision for the school and the use of SIG funding, the principal 
was described as unfocused, simply expressing a willingness to use the funds for any and all "legally 
allowable" purposes. SIG funding in Year 1 appeared to support a "business-as-usual" approach to 
addressing performance problems, with modest changes, if any, to prior activities. In Year 2 (2011- 
12), a new principal (who had previously taught at Gale) was welcomed by the faculty. Teachers cited 
his motivation, open-door policy, and instructional leadership as strengths, and teachers openly noted 
an upgrade in leadership compared with the previous principal. 

According to the new principal, his staff made "a major upgrade" in Year 2, explaining that he had 
"been able to transfer out two or three teachers that did not need to be here" and had made "major 
upgrades in those areas. ..that's going to pay off over the long run." Several members of the school 
improvement team reported that the quality of teachers was improving and that school 
administrators were conscientiously seeking staff that "fit" with the school. This perspective was 
shared by the ninth-grade academy coordinator, who explained that the principal was doing "a really 
good job" replacing staff and evaluating potential new staff members, adding that "He does a really 
good job corresponding with our LEA, our district office, about what he wants and what he needs to 
be successful in the school." 

Also of note with respect to staff skills, the school introduced a new policy of placing the weakest 
students with the strongest teachers as a means to leverage teacher strengths. During extended 
learning time (i.e., afterschool tutoring and Saturday Academy), for example, students rotate among 
teachers on the basis of the teachers' strengths. As expressed by the principal, "We're trying to be 
very prescriptive in what we do and trying to put the strongest teacher we have for the weaknesses 
they have. And we're doing that across the board." 

With regard to professional learning, teachers at Gale participated in fewer professional learning hours 
than did teachers in other schools, but there is evidence that professional learning opportunities were 
aligned with school needs and goals. Gale focused on three areas of professional learning: Common 
Core State Standards (reported by teachers, the principal, and the curriculum specialist), technology 
(reported by teachers and the principal), and opportunities for teachers to engage in depth in their 
subject areas (reported by the curriculum specialist and teachers). Areas for professional learning were 
aligned with district and principal priorities, as well as needs identified by teachers. For example, the 
instructional coach reported administering a survey to capture staff development needs. With the 
assistance of the SIG partner, the school developed a professional learning plan for literacy aligned with 
the district's push for literacy. 

In summary, through staff replacement, allocation of teachers' skills, professional learning, and the 
support of an external partner (among other supports), Gale appeared to make multiple efforts to build 
human capital (aggregate score of 7.5), most of which occurred in the second year of SIG. 
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Box 7.4. Raven Ridge Elementary's Efforts to Build Human Capital 

In contrast to Gale Secondary, Raven Ridge Elementary is located in an urban context that was 
categorized as traumatic (see Le Floch et al., 2014). Reports of violence in the community were 
common, as were instances of student neglect and teen pregnancy. The principal, new to the school 
in Year 1 of SIG (2010-11), commented that many of the parents of her students were teens 
themselves who had attended the elementary school fairly recently. Raven Ridge implemented the 
restart model and was being operated by an education management organization. This transition, 
decided upon by the school district, was extremely unpopular with the school community and the 
remaining school staff. The EMO (Raven Ridge's SIG partner) recruited and hired the new principal in 
the summer prior to SIG, removing a longstanding principal who was popular with the community. As 
a result, nine teachers requested voluntary transfers. The principal was also new to the district, which 
reportedly made her first year at the school much more challenging because she was required to 
abide by two new sets of policies and processes: those from the school district and those established 
by the SIG partner. 

Despite some teachers leaving Raven Ridge, many of the pre-SIG teachers had not changed. From the 
principal's perspective, the quality of her teachers was of primary concern. In an interview from 2011, 
she commented, "All the research says the teacher is the one that makes the most difference. We 
have maybe, honestly, six or seven, and that's generous, teachers on this campus that are truly 
competent." 

Given the principal's comments, one might expect that Raven Ridge was engaged in a variety of 
efforts to increase human capital. Flowever, with an aggregate score of 3.5, we did not find this to be 
the case. Although teachers reported participating in a high volume of professional learning activities 
(136 hours), these sessions were not described as effective by teachers. The principal, SIG partner, 
and instructional coaches all identified a number of efforts to build structures for data use (e.g., 
benchmark assessments in English language arts and math, student data systems supported by the 
district), but the teacher collaboration time that was needed to use these data structures effectively 
was often reportedly absent. In 2011-12, the principal, instructional coaches, teachers, and the SIG 
partner explained that time was set aside once a week for professional learning and teacher 
collaboration. Flowever, the coaches mentioned that collaboration time was not used consistently 
and that it was often shortened or usurped for other purposes. Teachers also reinforced that they did 
not get sufficient time to plan or collaborate. The apparent lack of distributed leadership at the 
school, as well as the minimally purposeful efforts to connect staffing and professional learning to 
school needs, appeared to make efforts to build human capital at Raven Ridge more challenging. 

In summary, Raven Ridge was situated in a school and district context that remained challenging 
during its first two years of SIG. The principal, new to the district and the school, attributed the 
school's history of low performance mostly to its incompetent teachers. Consequently, she focused 
her efforts at building human capital on providing professional learning opportunities and setting up 
structures to support collaboration and data use. These efforts were implemented imperfectly, with 
teacher collaboration time often replaced with other activities and professional learning activities 
perceived by many as ineffective or not executed well. 
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Chapter Summary 

SIG policy includes some mandates that are explicitly tied to assumptions about human capital in low- 
performing schools. The requirement to replace principals (turnaround and transformation model) 
assumes that existing principals do not have sufficient knowledge and skills and have been unsuccessful 
in creating high levels of organizational capacity. The requirement to replace teachers (turnaround 
model only) assumes that existing teachers are not able to do their jobs well. The encouragement of 
external partnerships (especially with the restart model) assumes that districts do not have the internal 
capacity to adequately support their persistently low-performing schools. 

However, the data from interviews and focus groups discussed in Chapters 3 to 6 reveal the complexities 
and unanticipated consequences of such mandates. Depending on the historical context of the school 
and the current policies and conditions, staffing transitions— both for teachers and principals— had 
unforeseen complications. Our analyses from these chapters also showed that new leaders were not 
always a good fit for the school, and they often left for other professional opportunities or for personal 
reasons. New, less experienced teachers sometimes created a positive culture shift, but district staffing 
policies also did not always protect these young educators from layoffs. Some districts were forced to 
replace underperforming teachers at a SIG school with equally poor teachers from another. Such actions 
reportedly created instability and potentially sabotaged capacity-building efforts. 

At the district level, administrators reported that changes to governance structures or new SIG positions 
allowed for an increase in the number of support services for SIG schools. However, fewer than half of 
SIG school principals in our core sample perceived these district supports as useful to the overall 
improvement process. Moreover, some school respondents explained that instability among district 
administrators hindered or stalled improvement processes underway in SIG schools. 

Although our study methodology does not permit inferences regarding actual improvement of teacher 
knowledge and skills, the survey data suggest that teachers felt they were learning from the professional 
learning activities in which they participated. Overall, teachers reported participating in large amounts 
of professional learning, both job-embedded and traditional, with many schools offering more hours 
than the national average for low-performing schools. Also, survey results showed that teachers in most 
core sample schools reported learning and changing practice in at least one content area. In schools 
where our qualitative data allowed us to understand whether the school was specifically targeting a 
certain type of professional learning, we frequently found that high percentages of teachers reported 
learning and changing practice in those areas. 

Overall, our data indicate that the 25 core sample schools engaged in many activities intended to build 
human capital, both in compliance with SIG requirements and as part of each school or district's own 
decisions about how to best address capacity issues. The majority of the schools in this sample sought to 
build human capital and to adhere to the requirements of the grant they had been awarded. In Part III of 
this report, we will explore the extent to which these efforts were associated with any apparent changes 
in organizational capacity by the end of SIG. 
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Part III: Change and Sustainability 

Chapter 8: Changes in Improvement and 
Organizational Capacity 


In this chapter, we focus on 12 of the 25 core sample schools, which we collectively refer to as the core 
subsample, 44 to address the following questions: 

1. Do teachers perceive their schools to have improved over the course of SIG? 

2. Have schools' levels of organizational capacity changed over the course of SIG? 

3. Are schools' efforts to build their human capital related to changes in organizational capacity? 

Building on Part II, which described the efforts schools reported putting forth to try to spur change, this 
chapter focuses on two leading indicators of change: perceived improvement and organizational 
capacity to improve student learning. The Part II measures can be considered inputs to the change 
process, while this chapter's measures can be considered intermediate outcomes in the change process. 
Actual measures of student achievement can be considered ultimate outcomes, but as noted in Chapter 
1, they are not a focus of this study. 


Box 8.1. Key Chapter 8 Findings 

• Most teachers in 7 of the 12 core subsample schools reported on the Year 3 teacher survey 
that their school had changed in primarily positive ways throughout the course of SIG. 

• The quality of leadership in the 12 core subsample schools appears to be related to teachers' 
perceptions of school improvement, based on data from the teacher survey and site visits. 

• In Year 3, five schools received a higher organizational capacity score, six received a moderate 
capacity score, and one received a lower capacity score based on our summative ratings of 
schools' organizational capacity. 

• A school's level of organizational capacity during the first year of SIG appears to be related to 
the amount of improvement in the school's level of organizational capacity from Year 1 to 
Year 3. Core subsample schools that had lower organizational capacity in Year 1 tended to 
improve their capacity by Year 3. Schools that had moderate or higher capacity in Year 1 
tended to maintain their moderate or higher level of capacity by Year 3. 

• Core subsample schools that appeared to make more effort to build human capital in Years 1 
and 2 of SIG were more likely to improve their capacity (or sustain their already higher 
capacity) than schools that appeared to make less effort to build human capital. 


44 We selected a subset of 12 of the 25 core sample schools in Year 3 to facilitate more in-depth analyses of the 
change process across all 3 years of SIG. See Chapter 2 for more details on the core subsample and related 
procedures. 
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Perceived Improvement 

SIG was intended to be a significant intervention for persistently low-performing schools, catalyzing 
dramatic action and yielding perceptible improvements. Although this study does not consider whether 
SIG improved student outcomes, we did collect qualitative and quantitative data on respondents' 
perceived improvement in elements of organizational functioning and educational practice during the 
three years of SIG (see Le Floch et al., 2014). These elements may reflect changing conditions in the 
schools over time, and studies have frequently found them to be associated with improved student 
outcomes (see Bryk et al., 2010; Herman et al., 2008). While perceptions do not necessarily correlate 
with more objective measures of change, they could influence how respondents subsequently relate to 
and participate in the improvement process. Teachers and administrators who perceive improvement 
may become more committed to reforms, increase their own perceptions of efficacy, and increase their 
motivation to sustain the improvement process. Such patterns are consistent with theory on employee 
motivation and organizational change (Lawler, 1994; Mohrman & Lawler, 1996). 

Drawing on teacher survey responses in Year 3 of SIG, we focus on respondents' perceptions of overall 
improvement over the course of SIG and explore school-level associations based on site visit data from 
district administrators, principals, teachers, instructional coaches, school improvement teams, and 
students. 

Overall Perceived Improvement in Core Subsample Schools 

The teacher survey administered in Year 3 of SIG (2012-13) asked teachers to assess the overall 
improvement in their schools with the following question: "Do you believe that your school has 
undergone changes over the past three years?" Teachers were asked to indicate if they felt that their 
school had changed in primarily positive ways, both positive and negative ways, primarily negative ways, 
or if the school had not changed much over the course of SIG. 

Most teachers in 7 of the 12 core subsample schools 45 reported that their schools had changed in 
positive ways (see Exhibit 8.1). Across all 12 schools, the average percentage of teachers who reported 
overall positive improvement was 53 percent, compared with 8 percent of teachers who reported 
negative changes overall. There was, however, variation across schools, with the percentage of teachers 
who reported that their school had changed in primarily positive ways ranging from 0 percent in one 
school to 97 percent in another. 


45 On the Year 3 teacher survey, teachers were asked to respond to the following question: "Do you believe that 
your school has undergone changes over the past three years?" Teachers who responded / do not know how the 
school has changed over the past three years were not included in these analyses. 
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Exhibit 8.1. 

Overall Perceived Improvement During SIG, by Core Subsample School, 2010-13 


Baltimore Bridge Elementary 
Meribel High 
Sterling Slope Elementary 
Gale Secondary 
Island Bay Elementary 
McAlliston High 
Elmsville High 
Paul Bunyan High 
Coral High 
Proctor Point High 
Blizzard Bay Elementary 
Sawbuck Elementary 

0% 20% 40% 60% 80% 100% 

■ School changed in positive ways overall □ School changed in negative ways overall 

□ School changed in both positive and □ School did not change much 

negative ways 

Source: SST teacher survey, spring 2013 

Notes: Includes 12 core subsample schools. All school names are pseudonyms. 



Overall Perceived Improvement and Principals' Leadership in Core Subsample 
Schools 

Leithwood et al. (2004) found that "there are virtually no documented instances of troubled schools 
being turned around in the absence of intervention by talented leaders.... While other factors within the 
school also contribute to such turnarounds, leadership is the catalyst" (p. 17). Data from the Year 3 
teacher survey and site visits do indeed suggest an association between the quality of principal 
leadership in these schools and teachers' perceptions of improvement (see Le Floch et al. [2014] and 
Exhibits B.21, B.22, and B.23 for more detail on the analyses of principal leadership). 

In Year 3 of SIG, teachers in all 3 of the 12 core subsample schools whose principals were reported as 
exhibiting higher levels of teacher trust and overall leadership (including transformational, 
instructional, and strategic leadership) also reported the highest levels of perceived improvement 
over the three years of SIG (see Exhibit 8.2; see Le Floch et al. [2014] for more detail on the analyses of 
principal leadership). Conversely, the four core subsample schools with the lowest aggregate ratings 
across our leadership measures were also the ones whose teachers reported the lowest levels of 
perceived improvement over the course of SIG. 
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Exhibit 8.2. 

Summary of Teachers' Perceived Improvement and Principals' Leadership, by Core 
Subsample School 



Percentage of 
Teachers 
Reporting 
Improvement 

Years of 
Principal 
Replacement 

YEAR 1 

Teacher 

Trust 

in Principal 

YEAR 1 

Overall 

Leadership 

Rating 

YEAR 3 

Teacher 

Trust 

in Principal 

YEAR 3 

Overall 

Leadership 

Rating 

Baltimore Bridge Elementary 

97 

Year 1 

High 

High 

High 

High 

Meribel High 

92 

Year 2 

Moderate 

Moderate 

High 

High 

Sterling Slope Elementary 

85 

Year 2 

High 

High 

Moderate 

High 

Gale Secondary 

76 

Year 2 

Moderate 

Low 

Moderate 

Moderate 

Island Bay Elementary 

67 

Year 1 

Year 2 

Low 

Low 

Moderate 

Moderate 

McAlliston High 

57 

No change 

High 

Moderate 

Moderate 

Moderate 

Elmsville High 

53 

Year 1 

Moderate 

Moderate 

Moderate 

Moderate 

Paul Bunyan High 

49 

Year 3 

Moderate 

Moderate 

Moderate 

Moderate 

Coral High 

43 

Year 1 

Year 2 

High 

Moderate 

Low 

Moderate 

Proctor Point High 

31 

No change 

High 

Moderate 

Moderate 

Moderate 

Blizzard Bay Elementary 

10 

Year 1 

Low 

Moderate 

Low 

Moderate 

Sawbuck Elementary 

0 

Year 1 

Year 3 

Moderate 

Moderate 

Low 

Low 


Source: SST teacher and instructional coach interviews, and teacher and school improvement team focus groups, spring 2011 
and spring 2013; SST teacher survey, spring 2011 and spring 2013. 

Notes: Includes 12 core subsample schools. All school names are pseudonyms. McAlliston High did not meet the 50 percent 
response rate threshold on the teacher survey in Year 1, so the classifications for the school's Year 1 indicators are based on a 
lower response rate and should be interpreted with this caveat. Due to insufficient data on transformational and instructional 
leadership, for two schools in Year 1 (Gale and Meribel), the leadership rating is based on the measure of strategic leadership 
only. 


Respondents from each of the three schools reporting the highest ratings of leadership quality in Year 3 
spoke to the principal's commitment to the school, staff, and students. For example, at Meribel High, 
which replaced its principal in Year 2, the sense of campus community appeared to stem from the 
principal's own dedication to the school. The instructional coach explained that the prior principal's view 
of his role was that "it's just a job, and it's just a stepping stone to get me somewhere else," whereas 
the current principal said, "It’s more like a calling. This is what I'm supposed to be doing. I'm supposed 
to be helping this site. I'm supposed to be doing as much as I can to support and get them off on the 
right direction." At Baltimore Bridge Elementary, the principal was new to the school in Year 1 and 
focused on building structures and relationships at the school. One district administrator described the 
principal as "young but phenomenal at motivating his staff. Fie is also really great at understanding the 
value of making everyone around him real drivers. We sometimes see principals who try to do 
everything. [Fie] has been strategic in how he uses his assistant principals, giving them big pieces to own 
and drive so that they can be more effective as a team." Respondents at Sterling Slope Elementary also 
spoke to the principal's level of commitment and dedication to the school. The school's instructional 
coach characterized the principal as "full of energy. Wanting to succeed. ..she wants all of us to succeed. 
She looks out for the school. Her heart and soul is in the school. She's the first one in and the last one 
out." 

In contrast, respondents at Sawbuck Elementary, a school with one of the lowest reported levels of 
perceived improvement over the course of SIG, described the new principal in Year 3 as a primary factor 
for the school's setbacks. In the first two years of SIG, this school was led by a principal who was rated 
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"mid-high" on strategic leadership (Le Floch et al., 2014). However, the principal who was new to the 
school in Year 3 was rated relatively low across various measures of leadership. One school 
improvement team member described a desire for the new principal to "take ownership": 

/4s a leader, take ownership of stuff that needs to happen. There are systems that used to exist 
that don't happen anymore.... As a leader, you are that point person to get it off the ground. It 
has been lazy this year, "It doesn't really matter." That is the motto this year. I don't feel a sense 
of caring— not from teachers, not from home, not from leadership. 

We similarly examined whether there were any apparent associations between perceived improvement 
and school level (elementary versus high school), SIG intervention model, external school context, or 
overall per-pupil SIG award levels, which did not yield any apparent associations. 

Changes in School Organizational Capacity 

In addition to looking at teachers' perception of improvement, we also examined how indicators of 
school organizational capacity changed over the course of SIG. Positive changes in capacity could be 
precursors to improved student outcomes and part of a sustained process of continuous improvement. 

"Developing and building capacity" is commonly cited as a strategy for helping low-performing schools 
improve (Fuhrman, Goertz, & Duffy, 2004; Mintrop & Trujillo, 2005), although the term capacity is used 
in different ways by researchers and policymakers alike. Some research focuses on expanding our 
understanding of teacher capacity to include the multidimensional aspects of human capital (which 
include teachers' skills, knowledge, and dispositions) as well as the broader facets of organizational 
capacity (such as the vision and leadership of a school, the collective commitment and cultural norms of 
the staff, the collective knowledge or access to knowledge of the organization, and the organizational 
structures and management of the school) that may impact the capacity of teachers (O'Day et al., 1995). 
Other researchers build on this notion of interconnectivity and posit a component-based definition of 
capacity, which consists of human capital, social capital, program coherence, and resources (Beaver & 
Weinbaum, 2012). Despite these differences, all existing frameworks portray school capacity as having 
multiple, interconnected attributes. 

Drawing on site visit data from district administrators, principals, teachers, instructional coaches, school 
improvement teams, parents, and students, our measure of organizational capacity focuses on eight 
leading indicators, or forms of capacity that schools can leverage to improve teaching and learning, 
namely the principal's leadership skills, the coherence among school strategies and goals , the presence 
of clear and shared goals, the prevalence of teacher collaboration structures and routines, the level of 
trust among teachers, the safety and orderliness of the school environment, the prevalence of data use 
structures and routines, and the extent to which school leaders and staff assume responsibility for the 
school's performance problems (see Box 8.2; see also Exhibit B.28 and Appendix D for further details on 
this aggregate measure and each indicator). We were unable to collect data on all potentially important 
leading indicators suggested by prior research— such as changes in teachers' knowledge and skills, 
changes in curricula and instruction, and level of resources, among others (Newmann et al., 1997; O'Day 
et al., 1995). Furthermore, our measure of organizational capacity is not an absolute measure of 
capacity, but rather a measure intended to capture capacity to improve student learning relative to the 
other schools in our core subsample. Even the higher capacity schools in our sample may have low 
capacity relative to a broader sample of schools. These caveats should be kept in mind when 
interpreting our findings in this section. 
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Box 8.2. Organizational Capacity in Core Subsample Schools 

Our measure of organizational capacity (see Le Floch et al., 2014) includes the following eight indicators: 

• Leadership (see Exhibits B.21-B.23), specifically the extent to which principals exhibit skills in 
providing transformational leadership, instructional leadership, and strategic leadership. The 
first two measures were based on survey and qualitative data from teachers, instructional 
coaches, and school improvement teams, the third solely on qualitative data from principals. 
Taking into account all dimensions, school leaders were classified as high, moderate, or low. 

• Coherence (see Exhibit B.28), or the degree to which the policies of a school reflect consistent 
goals; the strategies employed are clearly designed to foster achievement of these goals; and 
barriers and detractors from goals and strategies are systematically removed (Honig & Flatch, 
2004; Newmann et al., 2001). Categorizations for coherence were based on three teacher 
survey items related to programmatic coherence. For each item, schools were assigned numeric 
values based on the school mean relative to the overall mean, which were summed to create a 
coherence index, consisting of high, moderate, and low classifications. 

• Clear and shared goals (see Exhibit B.28), including the extent to which school leaders and staff 
have unity of purpose, explicit expectations, and shared values for student learning and success 
(Purkey & Smith, 1983; Newmann et al., 2001). Based on the shared goals survey scale, schools 
were classified as high, moderate, or low. 

• Teacher collaboration (see Exhibit B.24), often described in the literature as either same-subject 
teachers "identifying a common curriculum, and then analyzing common assessment data to 
make instructional changes" (DuFour, 2004b) or as teachers of the same students, but of 
different subjects, working together (Erb & Doda, 1989; Rottier, 2001). This measure is based on 
qualitative data from principals, teachers, and instructional coaches as well as teacher survey 
items concerning the extent to which teachers have established collaboration structures and 
routines. Based on both data sources, schools were classified as having a culture of 
collaboration, some collaboration, or inconsistent collaboration. 

• Teacher-teacher trust (see Exhibit B.28), or the extent to which teachers feel they have mutual 
respect for each other, for those who lead school improvement efforts, and for those who are 
experts at their craft (Consortium on Chicago School Research, 2004). Based on the teacher- 
teacher trust survey scale, schools were classified as high, moderate, or low. 

• Safe and orderly climate (see Exhibit B.25), or the presence of an environment in which 
students "have a sense of being physically and psychologically safe in their school" (Consortium 
on Chicago School Research, 2004, Student-Centered Learning Climate section). Based on 
qualitative data from district administrators, principals, teachers, instructional coaches, 
students, and parents, schools were classified as safe/orderly, mixed, or unsafe/disorderly. 

• Use of data for instructional decisions (see Exhibit B.26), characterized as the presence of 
structures and routines for monitoring student learning and engaging in frequent and 
transparent use of student outcome data to guide instructional decisions (Coburn & Beuschel, 
2012; Coburn & Turner, 2011a; Coburn & Turner, 2011b). Based on qualitative data from district 
administrators, principals, teachers, and instructional coaches, schools were classified as high, 
medium, or low. 
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Box 8.2. Organizational Capacity in Core Subsample Schools 

(continued from previous page) 

• Locus of responsibility (see Exhibit B.27), characterized as the extent to which school 
respondents attributed the performance problem in their school to factors within their 
control (i.e., internal causes) as opposed to outside of their control (i.e., external causes). 
Based on qualitative data from principals and teachers, schools were classified as having internal 
responsibility, limited internal responsibility, or external responsibility. 

For each indicator, we assigned numeric values to the classifications (0 for the lowest category, 1 for 
the middle category, and 2 for the highest category), which were summed to create an aggregate 
index of school capacity. School classifications on the overall level of organizational capacity are 
described below (see Exhibit B.28 for more detail on the analytic procedures). 

Higher capacity 

• Received a summative rating of at least 10 out of 16 on the school capacity index. 

Moderate capacity 

• Received a summative rating of 8 or 9 out of 16 on the school capacity index. 

Lower capacity 

• Received a summative rating of less than 8 out of 16 on the school capacity index. 


The summative ratings of capacity in the core subsample schools for Year 3 of SIG ranged from 7 to 15. 
Exhibit 8.3 displays overall capacity ratings for each school in Year 1 and Year 3 of SIG. It also shows the 
ratings separately for each of the eight leading indicators. 

Overall in Year 3, five schools received a higher capacity score, six received a moderate score, and one 
received a lower score. Within each rating category, schools varied among a number of characteristics, 
including school enrollment, school level (elementary, high), SIG model (transformation, turnaround, 
restart), and external context (benign, depressed, traumatic). 

We found two factors that appeared to be related to changes in the capacity of our core subsample 
schools over the course of SIG: 46 

• Initial capacity in Year 1. Core subsample schools that had lower organizational capacity in Year 
1 tended to improve their capacity by Year 3. Schools that had moderate or higher capacity in 
Year 1 tended to maintain their moderate or higher level of capacity by Year 3. 

• Efforts to build human capital in Years 1 and 2. Schools that appeared to make more effort to 
build human capital in Years 1 and 2 of SIG were more likely to improve their capacity (or sustain 
their already higher capacity) than schools that appeared to make less effort to build human 
capital. 


46 We examined several other factors that might be related to organizational capacity— including teacher 
replacement, principal turnover, principal-teacher trust, per-pupil SIG expenditures, and overall SIG award level — 
and found no apparent relationships. 
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Exhibit 8.3. 

School Classifications on Leading Indicators and Overall Organizational Capacity, by Core Subsample School, 
2010-11 and 2012-13 



Leading Indicator 

Overall 

Capacity 

Score 

Overall Capacity 
Rating 

Leadership 

Coherence 

Clear and 
Shared Goals 

Teacher 

Collaboration 

Teacher-Teacher 

Trust 

Safe and Orderly 
Climate 

Use of Data 

Locus of 
Responsibility 

Yrl 

Yr 3 

Yrl 

Yr 3 

Yr 1 

Yr 3 

Yr 1 

Yr 3 

Yr 1 

Yr 3 

Yrl 

Yr 3 

Yr 1 

Yr 3 

Yrl 

Yr 3 

Yrl 

Yr 3 

Yrl 

Yr 3 

Blizzard Bay 
Elementary 

Moderate 

Moderate 

Lower 

Moderate 

Lower 

Lower 

Some 

Some 

Moderate 

Moderate 

Unsafe and 
disorderly 

Mixed 

Lower 

Higher 

Limited 

internal 

Limited 

internal 

4 

8 

Lower 

Moderate 

Island Bay 
Elementary 

Lower 

Moderate 

Lower 

Lower 

Lower 

Moderate 

Some 

Some 

Moderate 

Moderate 

Safe and 
orderly 

Mixed 

Lower 

Higher 

Limited 

internal 

Limited 

internal 

5 

8 

Lower 

Moderate 

Elmsville High 

Moderate 

Moderate 

Moderate 

Moderate 

Moderate 

Moderate 

Inconsistent 

Inconsistent 

Moderate 

Moderate 

Safe and 
orderly 

Safe and 
orderly 

Lower 

Moderate 

Limited 

internal 

Limited 

internal 

7 

8 

Lower 

Moderate 

Gale 

Secondary 

Lower 

Moderate 

Moderate 

Higher 

Moderate 

Moderate 

Some 

Culture of 
collaboration 

Moderate 

Moderate 

Safe and 
orderly 

Safe and 
orderly 

Moderate 

Higher 

External 

External 

7 

11 

Lower 

Higher 

Coral High 

Moderate 

Moderate 

Moderate 

Moderate 

Moderate 

Moderate 

Some 

Some 

Moderate 

Moderate 

Safe and 
orderly 

Mixed 

Moderate 

Higher 

External 

Limited 

internal 

8 

9 

Moderate 

Moderate 

Sawbuck 

Elementary 

Moderate 

Lower 

Moderate 

Lower 

Moderate 

Moderate 

Some 

Some 

Moderate 

Moderate 

Mixed 

Mixed 

Higher 

Higher 

Limited 

internal 

Limited 

internal 

9 

7 

Moderate 

Lower 

Proctor Point 
High 

Moderate 

Moderate 

Lower 

Lower 

Moderate 

Moderate 

Some 

Some 

Moderate 

Moderate 

Safe and 
orderly 

Mixed 

Higher 

Higher 

Limited 

internal 

Limited 

internal 

9 

8 

Moderate 

Moderate 

Paul Bunyan 
High 

Moderate 

Moderate 

Moderate 

Moderate 

Moderate 

Moderate 

Culture of 
collaboration 

Some 

Moderate 

Moderate 

Mixed 

Safe and 
orderly 

Moderate 

Moderate 

Limited 

internal 

Limited 

internal 

9 

9 

Moderate 

Moderate 

McAlliston 

High 

Moderate 

Moderate 

Moderate 

Moderate 

Moderate 

Moderate 

Some 

Culture of 
collaboration 

Higher 

Moderate 

Safe and 
orderly 

Safe and 
orderly 

Moderate 

Moderate 

Limited 

internal 

Limited 

internal 

10 

10 

Higher 

Higher 

Meribel High 

Moderate 

Higher 

Moderate 

Higher 

Moderate 

Higher 

Some 

Some 

Moderate 

Higher 

Safe and 
orderly 

Safe and 
orderly 

Higher 

Moderate 

Limited 

internal 

Limited 

internal 

10 

13 

Higher 

Higher 

Baltimore 

Bridge 

Elementary 

Higher 

Higher 

Higher 

Higher 

Higher 

Higher 

Some 

Culture of 
collaboration 

Higher 

Higher 

Unsafe and 
disorderly 

Safe and 
orderly 

Higher 

Higher 

Internal 

Limited 

internal 

13 

15 

Higher 

Higher 

Sterling Slope 
Elementary 

Higher 

Higher 

Higher 

Higher 

Higher 

Moderate 

Culture of 
collaboration 

Culture of 
collaboration 

Higher 

Moderate 

Safe and 
orderly 

Safe and 
orderly 

Higher 

Higher 

Internal 

Internal 

16 

14 

Higher 

Higher 


Source: SST respondent interviews and focus groups, spring 2011 and spring 2013; SST teacher survey, spring 2011 and spring 2013. 

Notes: Includes 12 core sample schools. McAlliston did not meet the 50 percent response rate threshold on the teacher survey in Year 1, so the classifications for the school's 
Year 1 indicators are based on a lower response rate and should be interpreted with this caveat. Due to insufficient data on transformational and instructional leadership, for 
two schools in Year 1 (Gale and Meribel), the leadership rating is based on the measure of strategic leadership only. All school names are pseudonyms. 
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Initial Organizational Capacity in Year 1 

Overall, the organizational capacity exhibited by the 12 core subsample schools in Year 3 appeared to 
be related to their organizational capacity in Year 1 (see Exhibit 8.4). All four of the schools with lower 
capacity in Year 1 improved to moderate or higher organizational capacity by Year 3. Among the four 
schools with moderate capacity in Year 1, three remained moderate by Year 3. Similarly, among the four 
schools with higher capacity in Year 1, all four remained higher by Year 3. Thus, it appears that lower 
capacity schools tended to improve their capacity over time, while moderate and higher capacity 
schools tended to maintain a relatively more stable level of capacity. Below, we further discuss the 
trajectories of schools at both the ends of the spectrum. 


Exhibit 8.4. 

Organizational Capacity in Year 1 Relative to Organizational Capacity in Year 3 



Lower Organizational 
Capacity in Year 3 

Moderate Organizational 
Capacity in Year 3 

Higher Organizational 
Capacity in Year 3 

Total 

Lower 

Organizational 
Capacity in 

Year 1 


3 schools 

Blizzard Bay Elementary 
Island Bay Elementary 
Elmsville High 

1 school 

Gale Secondary 

4 schools 

Moderate 
Organizational 
Capacity in 

Year 1 

1 school 

Sawbuck Elementary 

3 schools 

Coral High 

Proctor Point High 

Paul Bunyan High 


4 schools 

Higher 

Organizational 
Capacity in 

Year 1 



4 schools 

McAlliston High 

Meribel High 

Baltimore Bridge Elementary 
Sterling Slope Elementary 

4 schools 

Total 

1 school 

6 schools 

5 schools 

12 

schools 


Source: SST respondent interviews and focus groups, spring 2011 and spring 2013; SST teacher survey, spring 2011 and spring 
2013. 

Notes: Includes 12 core subsample schools. All school names are pseudonyms. 


The finding that higher capacity schools were able to at least maintain their organizational capacity is 
consistent with studies that have demonstrated that existing capacity begets capacity (Beaver & 
Weinbaum, 2012; Opfer & Peddler, 2013). This view is reflected in many state and federal grants for 
low-performing schools, which require that eligible schools demonstrate a minimum level of capacity, or 
"readiness to benefit" (Le Floch & Boyle, 2006). Although our finding is not surprising given this context, 
it is still relevant to ask how higher-capacity schools in our core subsample were able to at least 
maintain a relatively higher level of capacity over the course of SIG. 

One hypothesis focuses on the coherence of multiple, interconnected supports for school change. 
Researchers hypothesize that it is not enough for a school to have isolated efforts to build and sustain 
capacity (Beaver & Weinbaum, 2012; Corcoran & Goertz, 1995). For example, it is not sufficient to have 
a highly qualified team of teachers if the school lacks effective leadership and a coherent approach to 
capacity-building efforts. If a school is to succeed in building and sustaining capacity, it ideally needs to 
exhibit multiple, interconnected features of capacity that collectively help to bring about change. Three 
of the four schools in our core subsample that were categorized as higher capacity in Year 1 appeared to 
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adopt such a multifaceted, interconnected approach to improvement and maintained this approach 
over the course of SIG. 

For example, respondents from one of these schools described visionary leaders who rallied the staff 
around a mission. They also reported that there were structures within the school that allowed for 
capacity building to take place, such as weekly planning time, weekly data chats, programs that 
maximized instructional time, and new technology. There was a reported culture of collaboration and 
high levels of teacher-teacher trust. Improvement efforts were described by teachers as coherent and 
primarily focused on the instructional core. One teacher at the school described efforts as follows: 

[The school leaders] are really serious about improving the school. They're pretty ambitious. 

And, the team that's been put together to bring [the vision] about is very focused on bringing 
that to a reality, and it's all about raising the rigor... [there is] a focus on moving beyond the 
procedural and the memorization— [in] actually getting kids engaged with the concepts and the 
content. I think all the programs. ..it's meant to all work together, complement each other, 
striving for that improvement. 

Multiple efforts to build capacity within this school— which included hiring the appropriate leader, hiring 
the appropriate staff, providing coaching and professional learning opportunities to teaching staff, 
establishing the necessary infrastructure, and focusing on coherence— appeared to converge 
simultaneously during all three years of SIG, which may have helped the school to maintain relatively 
higher levels of capacity. 

On the other end of the spectrum, it is relevant to ask how lower capacity schools were able to improve 
their capacity ratings over the last two years of SIG. One hypothesis is that changes or interventions in 
one or more areas within a school may have the potential to influence changes in other areas of 
organizational functioning (Axelrod & Cohen, 1999). For example, in three of the four core subsample 
schools with lower capacity ratings in Year 1, a new leader appeared to play a major role in triggering 
changes within the school in subsequent years of the grant. These changes in leadership appeared to 
have had ripple effects in the trajectory of the school, which may help explain why these schools shifted 
from the lower to the moderate or higher categories by the final year of SIG. 

For example, one of these schools had a reportedly turbulent first year of SIG. Teachers noted that SIG 
had been forced upon them, and many of them felt betrayed by the district, which applied for the grant 
despite opposition. The district hired an unpopular principal, who left midway through the school year. 
Teachers described an assortment of unrelated improvement actions that were either ineffective or did 
not clearly tie into an overarching vision for school improvement. The assistant principal summed up the 
first year: “It felt like things were being done to people versus done with people." In the second year of 
SIG, there was a significant shift at the school. The district hired a new principal and replaced more than 
half of the teaching staff. The new principal viewed her role as the "keeper of the vision to keep it on 
track to make sure we have aligned coherence." She brought about changes that reportedly resulted in 
a culture shift. As one example, she expanded the leadership structure of the school to include more 
teachers and to reflect a more distributed style of leadership. With these changes, morale at the school 
improved by the final year of the grant. Teachers also reported a higher level of clear and shared goals 
than in Year 1, as well as higher levels of data use. Although the assistant principal mentioned that there 
is still work to do, this school appeared able to mobilize some aspects of capacity building. 

Organizational Capacity and Efforts to Build Human Capital in Years 1 and 2 

SIG emphasizes building capacity through changes related to governance, structure, practice, and in 
particular human capital "to effect rapid and substantial improvement of persistently low-achieving 
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schools" (Perlman & Redding, 2011, p. 5). For example, the turnaround model mandates that the school 
replace the principal and at least 50 percent of the staff in an effort to remove less-qualified teachers 
and leaders. Given the importance SIG places on building human capital, we also explored the 
relationship between schools' organizational capacity and efforts to build human capital. Our data 
suggest that schools' reported efforts to build human capital in the first two years of SIG (as reported in 
Chapter 7) is related to the likelihood that schools' organizational capacity improved over the three 
years of SIG (see Exhibit 8.5). 


Exhibit 8.5. 

Level of Effort to Build Human Capital in Years 1 and 2 Relative to Change in 
Organizational Capacity from Year 1 to Year 3 



Capacity Decreased 

Capacity Remained Stable 

Capacity Increased 

Total 

Fewer efforts 

to build 

human 
capital 
(2-5-5. 5) 

1 school 

Sawbuck Elementary 

3 schools 

Coral High 

Proctor Point High 

Paul Bunyan High 

1 school 

Elmsville High 

5 schools 

Many efforts 
to build 

human 
capital 
(6. 5-8. 5) 


4 schools 

McAlliston High 

Meribel High 

Baltimore Bridge Elementary 
Sterling Slope Elementary 

3 schools 

Blizzard Bay Elementary 
Island Bay Elementary 
Gale Secondary 

7 schools 

Total 

1 school 

7 schools 

4 schools 

12 schools 


Source: SST respondent interviews and focus groups, spring 2011 and spring 2013; SST teacher survey, spring 2011 and spring 
2013. 

Notes: Includes 12 core subsample schools. All school names are pseudonyms. 

See Exhibit 7.1 for measures of efforts to build human capital and Exhibit 8.3 for measures of organizational capacity. Each 
school's organizational capacity was classified as lower, moderate, or higher in Year 1 and in Year 3 of SIG. A school's capacity 
decreased if their classification decreased from Year 1 to Year 3 (e.g., moderate in Year 1, lower in Year 3), remained stable if 
their classification was the same in Year land Year 3, and increased if their classification increased from Year 1 to Year 3 (e.g., 
lower in Year 1, higher in Year 3). This analysis may be limited due to potential ceiling effects, as schools identified as having 
higher organizational capacity in Year 1 could not increase their capacity rating any further, and thus were placed in the stable 
capacity category. 


Schools that appeared to make more effort to build human capital in Years 1 and 2 of SIG were more 
likely to improve their capacity (or sustain their already higher capacity) than schools that appeared to 
make less effort to build human capital. Three of the seven core subsample schools with a human 
capital building rating of 6.5 or higher improved their organizational capacity rating from Year 1 to Year 
3. One of these schools had lower organizational capacity in Year 1 and higher organizational capacity in 
Year 3. This school received a new principal in the second year of SIG and subsequently invested in 
building human capital at the school. Respondents reported a focus on building infrastructures for data 
use, collaboration, and distributed leadership. When deciding between spending funds on human 
resources versus other types of resources for the school (such as technology), the principal stated, "I'm 
going to opt for the human resources piece because we believe that is, without a doubt, the basis for 
our success." The remaining four schools with a human capital building rating of 6.5 or higher already 
had higher organizational capacity ratings in Year 1 and were able to maintain those higher ratings in 
Year 3. 

Among the five core subsample schools that reported fewer efforts to build human capital in Years 1 
and 2 of SIG (human capital building rating of 2.2 to 5.5), one increased organizational capacity in Year 3 
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and three remained stable, while one decreased organizational capacity. Respondents from one of these 
schools in Year 1 described the administration's approach to SIG as "business-as-usual." That approach 
apparently continued: in Years 2 and 3, respondents did not report engaging in any activities that were 
judged to constitute "visible breaks" from prior practice, despite a new principal in Year 3. The school 
also reportedly made few efforts to build infrastructure for data use, teacher collaboration, or 
distributed leadership. It is thus not too surprising that this school showed no improvement in its 
organizational capacity rating from Year 1 to Year 3. 

School Stories of Change 

Although the aggregate ratings presented in this chapter provide a tool for comparing schools and 
identifying potential relationships between measures, the individual context of each school should also 
be considered to better understand perceptions of improvement and changes in organizational capacity. 
This section includes the stories of two contrasting schools that highlight some of the nuances and 
complexities of how these changes may occur. Both are stories of restart schools that had a "fresh start" 
in Year 1 of SIG and sought to build capacity in similar ways. However, one of these schools (Sawbuck 
Elementary, Box 8.3) ended up with a lower level of organizational capacity in Year 3, while the other 
(Baltimore Bridge Elementary, Box 8.4) ended up with a higher level. 


Box 8.3. Sawbuck Elementary: Obstacles to Growing Organizational Capacity 

Sawbuck Elementary's efforts to build capacity to improve student learning were reportedly stymied by 
leadership instability, an inexperienced teaching staff and external support organization, and a lack of a 
vision and change strategy. SIG coincided with a broader district reform initiative in which the district 
assigned a council of parents and community members to plan aspects of the change process at 
Sawbuck Elementary. In the first two years of SIG, as part of implementing the restart model, the 
school's leaders hired an entirely new teaching and administrative staff, integrated the help of an 
external support organization, and moved into a renovated facility. The school also reportedly put forth 
various efforts to build an infrastructure for collaboration and data use, and implemented a variety of 
professional learning opportunities. 

Despite the new resources and efforts to build human capital and organizational capacity, the school 
faced a variety of challenges. First, the new teaching staff included mostly first-year teachers who 
reportedly struggled with the challenges of a persistently low-performing school. Second, the members 
of the administrative staff— including the principal and assistant principals— were also new to their 
roles. High turnover among teachers and assistant principals was reported after the first year of SIG. 

In the second year, morale improved slightly as staff adjusted to their roles. However, the principal was 
perceived as disorganized and unfocused, and a lack of growth in student achievement prompted the 
CMO to terminate the principal. The departure of the principal was described as jarring and abrupt, 
casting a shadow over the school in its final year of SIG. In addition, one of the assistant principal 
positions was terminated, and another assistant principal was let go. 

By Year 3, respondents could not identify areas where the school had improved. Most teachers noted 
that the moderate progress they had made in the prior year had not continued into the final year of SIG, 
and in fact, many perceived the school to have regressed. The lack of progress was attributed by and 
large to a change in the leadership. Respondents felt that these changes were for the worse, as the new 
principal was not perceived as a strong leader or communicator. 
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Box 8.4. Baltimore Bridge Elementary: A Human Capital Focus 

The implementation of SIG at Baltimore Bridge Elementary coincided with a larger district initiative in 
which the district assigned the lowest-performing schools to an intervention. As part of the restart 
model, all school staff and leaders had been replaced, a new curriculum and new data-driven decision 
strategies were immediately implemented, and the school partnered with a CMO to help with the 
change process. 

In its first year of implementation, the CMO immediately set forth a "human capital focus," which 
included intensive coaching cycles, two hours of grade group meetings, professional learning activities 
once a week, formal observations of teachers multiple times throughout the year, and performance- 
based compensation. The school also implemented a longer school day and year, and the curricular 
approach included a focus on reading. 

In subsequent years of SIG, the school continued to focus on the strategies that had been set forth in 
the first year of implementation. The school maintained its focus on improving reading and expanded 
some of its strategies to encompass more students and classrooms— in addition, the school offered 
targeted teacher coaching on reading strategies. The principal, who was described as visionary and 
enthusiastic, made a concerted effort to transfer some of the leadership responsibility to the leadership 
team. 

In the final year of implementation, the school was reportedly a "teacher- and student-driven school." 
Teachers led professional learning activities and were encouraged to participate in planning meetings. 
Although there was some evidence of staff burnout by the end of SIG, the school appeared to continue 
on its trajectory toward change, and leadership was satisfied with the low rates of teacher turnover. The 
positive changes observed at Baltimore Bridge over the course of SIG were unanimously reported by 
school staff. In this sense, the story of Baltimore Bridge is one of a visionary leader who was aided by an 
external organization that placed a strong and consistent emphasis on instructional change and supports 
for building human capital. 


The two stories of Sawbuck Elementary and Baltimore Bridge Elementary reinforce our earlier points 
about the relevance of the interconnected aspects of capacity and the importance of strong leadership. 
At Sawbuck Elementary, the efforts to build human capital through data use and teacher collaboration 
were apparently thwarted by a variety of challenges that kept the school from leveraging its potential 
and expanding on the infrastructure it had created. Meanwhile, Baltimore Bridge Elementary was able 
to establish a variety of mechanisms for school change that appeared to support one another. The 
principal and CMO had a vision of how they would enact change, and they implemented improvement 
strategies accordingly by focusing on building human capital and instructional change related to reading 

Chapter Summary 

This chapter presented analyses related to perceived improvement over the course of SIG and changes 
in school-level capacity from Year 1 to Year 3 of the grant period. While restricted to the small set of 12 
core subsample schools, the findings support some tentative conclusions about the improvement 
process under SIG. First, most teachers in most schools perceived improvement overall during the three 
years of SIG, although the quality of leadership appeared to be related to the amount of perceived 
improvement. Although these perceptions do not necessarily imply actual improvements in student 
achievement, the overall positive trend may be a promising indicator for the school improvement 
process after SIG. 
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Second, there is some evidence that many of our core subsample schools were able to improve their 
levels of organizational capacity over time, or at least maintain a higher level of existing capacity. We 
also found that schools that improved their capacity or maintained a higher capacity were those that 
appeared to make a greater effort to build human capital in Years 1 and 2 of SIG. Again, the indications 
of improvement in organizational capacity do not necessarily imply actual improvements in student 
achievement, but they are perhaps an encouraging first step. 

SIG is meant to be a one-time infusion of resources in low-performing schools, and in this chapter we 
focused on perceived changes over the three-year grant period. However, SIG is also meant to lay the 
foundation for improvement that can be sustained well beyond the course of the grant. Whether or not 
our 12 core subsample schools have been able to successfully build this foundation for sustainability 
after SIG is a question to which we now turn in the following chapter. 
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Chapter 9: Sustainability of the Improvement 
Process 


SIG intended to provide a one-time infusion of resources to stimulate dramatic change that could be 
sustained beyond the three-year grant period. While discussions of sustainability often focus on the 
financial aspects of continuing specific interventions, sustaining the school turnaround process is more 
complex than, for example, finding money to maintain an instructional practice, extended day, or 
professional learning opportunities. As Jerald (2005) argued, sustainability entails maintaining, 
extending, and adapting improvement efforts in an unremitting reflective process. Fullan (2003) 
similarly posited, "Sustainability involves transforming the system in a way that the conditions and 
capacity for continuous improvement become built in" (p. 91). This notion of processes being built in or 
internalized is often cited as a definitional component of sustainability (Coburn, 2003). Thus, distributing 
leadership and building human capital or school organizational capacity more generally— topics we 
examined in earlier chapters— appear to be central components of sustainability. Some researchers 
have indeed reported a relationship between capacity and sustainability (Florian, 2000; Taylor, 2006). 
Flowever, there remain risks for even the schools that developed higher capacity by Year 3 of SIG. If a 
critical mass of key staff leave, they may take with them vital knowledge accumulated over time. If time 
for collaboration is cut, staff may not continue professional dialogue that fosters learning. These and 
other risks can compromise a school's overall prospects for sustainability. 

In this chapter, we explore how respondents in our 12 core subsample schools felt about their school's 
prospects for sustainability. First, we examine the relationship between organizational capacity and 
respondents' sustainability predictions. Next, we describe the challenges for sustainability that 
respondents believed their schools have confronted or will confront with the conclusion of SIG. Because 
data collection activities concluded with the three-year grant, we cannot document post-SIG activities. 
However, the study team did ask respondents about plans for replacing SIG funds, whether they 
anticipated continuing improvement actions, and concerns about threats to sustainability. 


Box 9.1. Key Chapter 9 Findings 

• Core subsample schools that had higher levels of organizational capacity were also the 
schools with higher scores on the teacher survey scale measuring perceived sustainability. 

• The most frequently reported risk factors for sustainability were related to human capital. For 
example, respondents in 9 of the 12 schools expressed concern about losing staff in the 
following school year, and respondents in 2 of the schools explicitly linked sustainability 
concerns to an impending change in school leadership. 

• Of the 12 schools we followed for all three years of SIG, 2 appeared to have strong prospects 
for sustainability, 6 appeared to have mixed prospects, and the remaining 4 appeared to have 
weak prospects, according to data from the teacher survey and site visits. 

• Schools' prospects for sustainability appear to be unrelated to several variables that might be 
hypothesized to predict sustainability, such as the size of the school's grant relative to 
baseline spending, enrollment, school level, or type of SIG intervention model. 

• Respondents in 2 of the 12 core subsample schools provided evidence of ownership of the 
improvement process by Year 3 of SIG. 


106 



Case Studies of Schools Receiving School Improvement Grants: Final Report 


Organizational Capacity and Perceived Sustainability 

In the previous chapter, we identified a set of indicators that collectively comprise our measure of 
organizational capacity (see Exhibit 8.3). These indicators include principal leadership, coherence, clear 
and shared goals, teacher collaboration, teacher-teacher trust, safe and orderly climate, use of data, and 
locus of responsibility. We posit that a school's organizational capacity constitutes a set of reserves that 
may facilitate the transition to the post-SIG era. As schools build their organizational capacity, they put 
in place the supports and strengths necessary to continue the improvements they have made, even as 
urgency fades and grants conclude. In predicting a school's prospects for sustaining improvements, we 
considered the overall level of organizational capacity, a measure that was examined in the previous 
chapter. 

In addition, we more directly measured perceived sustainability. In the spring 2013 teacher survey, 
respondents were given a series of Likert-scale items that asked them about their level of agreement 
with statements regarding the school's future (strongly disagree = 1, strongly agree = 4). Together, the 
following items scaled with a reliability of 0.87: 47 

a. School leaders (e.g., principal, department chairs) are committed to continuing the efforts to 
change this school for the better. 

b. Teachers in this school are committed to continuing the efforts to change this school for the 
better. 

c. Our school has the systems in place to sustain the changes we've made. 

d. School leaders (e.g., principal, department chairs) will have the resources (e.g., capacity, 
support) to continue the efforts to change this school for the better. 

e. Teachers in this school will have the resources (e.g., capacity, support) to continue the efforts to 
change this school for the better. 

We determined schools' perceived sustainability index by calculating the average score across all five 
survey items for the school. Across the 12 core subsample schools, the perceived sustainability index 
ranged from 2.58 to 3.53. To highlight some of the context surrounding these scores, Box 9.2 and Box 
9.3 present the stories of two contrasting schools— one that scored comparatively high on the perceived 
sustainability index (Sterling Slope Elementary) and one that scored comparatively low on the index 
(Blizzard Bay Elementary). 


47 The surveys were designed to measure constructs pertaining to the school as a whole related to contextual 
influences, selected improvement actions, and dimensions of implementation (see Exhibit 1.2). We assessed the 
quality of the teacher survey scales by conducting a confirmatory factor analysis on the items separately for each 
scale and by computing the scale reliability (Cronbach's alpha). Exhibit C-l in Appendix C shows the reliability and 
contributing items for each scale. Scale scores were computed on the basis of the mean of the individual items 
composing each scale. 
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Box 9.2. Sterling Slope Elementary: Commitment to Sustain Progress 

Sterling Slope Elementary's turnaround efforts began the year prior to SIG (2009-10) when the 
district assigned the school a new principal with a track record of improving academic performance. 
Described as a visionary leader, the new principal initiated a multi-faceted reform process that 
continued through the three years of SIG. The principal replaced over 50 percent of the school staff 
[in both the year prior to SIG and in Year 1 of SIG (2010-11)] in an effort to replace weaker teachers 
with stronger ones, overhauled numerous school processes (from curriculum to dismissal 
procedures), and instituted a variety of improvement strategies, many of which were developed and 
supported by the local school district. For example, the principal strengthened data use, established 
weekly collaborative planning time led by instructional coaches, and created opportunities for 
teachers to observe one another's instruction. 

Although the principal moved to another school in Year 2 of SIG (2011-12), the district promoted a 
highly-regarded assistant principal from the school to assume the principal position in Year 2, in what 
respondents universally characterized as "a seamless transition." The school also retained its "rock 
star" instructional coaches, who were familiar with the school's vision and could build on the practices 
they had established the previous year. In addition, district officials continued to conduct quarterly 
visits to the school to monitor progress and assess the needs of the school. With school leaders 
committed to continuing improvement efforts in place, school leaders and teachers reported 
deepening their implementation of improvement strategies in Year 2 with a staff that was "strong," 
"united," and fully on board with the school's vision for improvement. As one teacher commented, 

[At Sterling Slope] there's continuity and consistency.... Often in education reform, something 
is tested one year and if it didn't quite work, chuck it completely and try something new. What 
I've seen here is the opposite, [we're] taking lessons from failures.... We examine the 
approaches that worked [so we can] build on those strengths. 

Flowever, Sterling Slope still faced some challenges. Staff instability was a recurrent theme at Sterling 
Slope, due to the school's reliance on Teach for America (TFA) teachers. To address the waves of new, 
inexperienced teachers, district and school administrators strove to ensure a strong and stable 
leadership team. The school's principal and instructional coaches, who were trained to work explicitly 
with new teachers, remained consistent for the remainder of SIG and were expected to continue their 
roles even after the end of SIG. Likewise, district officials indicated that the district was committed to 
maintaining the support infrastructure it built during SIG (e.g., system of instructional coaches, 
monitoring visits) using other funds. As one of these instructional coaches concluded, 

There's been enough key players in the school who have seen the changes and the 
metamorphosis.... I think they have the knowledge and wherewithal to say this is how it's done 
[at Sterling Slope], So I think there are enough people here, in the various grade levels to be 
able to move forward. 
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Box 9.3. Blizzard Bay Elementary: An Uncertain Future 

In the years leading up to SIG, Blizzard Bay Elementary had seen numerous reform efforts come and 
go, and teachers— most of whom had worked in the school for years (and, in some cases, decades)— 
were reportedly feeling burned out by the frequent waves of reform efforts. With this in mind, the 
district's goal in writing its SIG application was to propose non-intrusive activities that minimized the 
burden placed on teachers, such as afterschool supports and additional counseling for students. 
However, SIG coincided with the district's adoption of a whole school reform model, which required 
widespread changes to school and classroom practice, including more student-centered approaches 
to instruction, increased data use, and new teacher leadership roles. 

Per SIG requirements, the district replaced Blizzard Bay's long-standing principal with a new leader, 
who was viewed as "charismatic" and "energetic" and had experience turning around low-performing 
urban schools. In Year 1 of SIG (2010-11), the principal established new governance structures, 
organized teacher professional development opportunities, and met regularly with teachers to 
prepare them to implement the whole school reform model in Year 2 (2011-12). At the beginning of 
Year 2, teachers reported feeling overwhelmed by the reform model's requirements, but by the 
spring, they described feeling more accustomed to the new practices. At the end of Year 2, district 
and school leaders noted several areas of school progress— including a stronger professional culture 
among teachers, higher expectations for students, improved instructional practices, and improved 
student performance— but they also acknowledged that more progress was needed. 

Year 3 of SIG (2012-13) brought significant changes. Due to district re-zoning, the school received an 
influx of new students that increased its total enrollment by nearly one third, and class sizes soared. 

In addition, the district assigned the principal additional responsibilities supporting other schools in 
the district, which reduced the time and attention the principal could devote to Blizzard Bay. Teachers 
continued to focus on implementing the whole school reform model in Year 3, but their efforts were 
reportedly overshadowed by an upsurge in student behavior problems. By the spring, teachers 
described a chaotic school environment with student discipline at an all-time low. Respondents 
attributed the behavior problems to various factors— overcrowding at the school, students' 
challenging home environments, students' boredom with the new reform model— and despite efforts 
to collect and analyze data on student behavior, teachers reported no successes or improvements in 
that area. 

By the end of Year 3, respondents described an uncertain future for Blizzard Bay. District officials 
planned to continue implementation of the whole school reform model for at least the next few 
years. They did not, however, have a plan for addressing the school's overcrowding, and ongoing 
budget pressures threatened the school's access to sufficient resources to continue improvement 
efforts. Teachers reported feeling defeated in the face of student behavior problems, and there was a 
possibility the school would lose its principal to another school or a position in the district office. 


Core subsample schools that had higher levels of organizational capacity by Year 3 of SIG (2012-13) 
were also the schools with higher scores on the teacher survey scale measuring perceived 
sustainability. Exhibit 9.1 plots for each school the organizational capacity measure in Year 3 of SIG (first 
presented in Chapter 8) against the sustainability index based on the teacher survey (described above). 
The plot shows that schools with higher organizational capacity also tended to have higher sustainability 
scores. This finding is consistent with the hypothesis presented above that capacity is a central 
component of sustainability. 
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Exhibit 9.1. 

Perceived Sustainability Index and Organizational Capacity, 2012-13 
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Sources: Sustainability index: SST teacher survey, spring 2013. Organizational capacity index: principal, instructional coach, and 
teacher interviews; teacher, leadership team, and community focus groups; SST teacher survey, spring 2013. 

Notes: Includes 12 core subsample schools. All school names are pseudonyms. 


Risk Factors 

Having a high level of organizational capacity when SIG ends may be a necessary condition for 
sustainability but perhaps not a sufficient one. For example, if leadership instability ensues, teachers 
depart, staff positions are cut, professional learning is discontinued, or funding is cut, then 
institutionalized knowledge or practices may quickly dissipate (Berends et al., 2001; Bodilly, 1998; 

Taylor, 2006). 

Anticipated Turnover in School Staff 

Respondents in 9 of the 12 core subsample schools expressed concern that losing staff would be a 
threat to sustainability (see Box 9.4). In five of these schools, respondents anticipated losing staff whose 
positions were funded through SIG, including instructional coaches, dropout prevention staff, and 
community or parent-community coordinators. As a teacher from one school explained, "I worry about 
positions disappearing after SIG because I don't know what we are going to do without those. Some of 
our ability to meet those needs or that vision has come through those additional positions." The 
principal of another school remarked: 

Obviously, another challenge with SIG is that we won't retain all of our positions. So, like our 
math coach, our math acceleration teacher, our community schools coordinator, our ELD coach, 
we won't have some of those positions.... We just have to be strategic about how we use our 
human capital. 

Respondents in five of these schools also explained that they were likely to be able to retain some, but 
perhaps not all, of the staff who had been newly hired through SIG. The principal at one school clarified, 

The central office— even though it's not at the same [funding] level as [under] SIG— they are 
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centrally funding positions, so they have identified schools for intensive support, strategic 
support.... The superintendent is keeping his word, and they are providing us with resources. 


Box 9.4. Risk Factors for Sustainability in Core Subsample Schools 

This analysis focuses on four specific perceived challenges to sustaining and continuing school 
improvement, including (1) anticipated turnover or loss of staff; (2) change in school leadership; (3) 
lack of district support, particularly with regard to retaining principals and teachers; and (4) loss of 
specific interventions (such as professional learning or extended day programs). Identification criteria 
for perceived risk factors are based on Year 3 interview and focus group data (see Exhibit B.29 for 
more detail on the analytic procedures). For this analysis, respondents refer to individuals from the 
following respondent groups: district administrators, principals, teachers, instructional coaches, 
external support providers, and school improvement team members. 

Identified as a perceived challenge for sustainability 

• Respondents from at least three respondent groups described this explanation as a 
sustainability challenge. 


In five schools, respondents were concerned about general staff turnover associated with schools in 
challenging contexts. Among these schools was a high school, at which a quarter of the teaching staff 
turned over each year, according to respondents. Respondents at a second school reported that while 
its faculty had stabilized during SIG, concerns lingered that teachers would not be willing to make the 
long drive to the school once they lost the extended day program and the associated pay (about 15 
percent additional pay). As a teacher from a third school explained: 

We have a lot of teachers who— they're just tired. Working in schools like this is exhausting.... 

The things that you see here, the hours, the work, the exhaustion, it's a lot. You're going to have 
administration changes next year. Our athletic director is moving on. We might have [an] 
activities director change, our head football coach just got promoted to an AP, and so that's 
another major change there. There is going to be a lot of changes. 

Across these schools, respondents had similar assessments of the role of human capital in sustaining the 
school improvement process. As the principal of one school explained, "at the end of the day it's not 
about the programs and it's not about having anything else, but it's about having the right people in the 
right place." 

Anticipated Change in Leadership 

Many of the core subsample schools had a high rate of principal turnover over the course of SIG, with 5 
of the 12 schools replacing their principals twice during the three-year grant period. At our site visits in 
spring 2013, we asked respondents whether they anticipated a new principal in fall 2013, and indeed, 
respondents in 4 of the 12 schools said that they did. 

Interview and focus group respondents in two schools explicitly linked concern about the prospects 
for sustainability to the impending loss of the current principal (see Box 9.2). The principal of one of 
these schools exhibited the strongest leadership skills (see Exhibit 8.2), and it was in this school that 
respondents voiced the most concern about the principal's departure. As one teacher commented, 
"Everything changes when there's a new principal. Everything changes. It doesn't matter how much the 
principal put together before leaving." In two other schools where respondents reported anticipating a 
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new principal in fall 2013, the quality of the departing leadership was not as high, but respondents 
described the turnover as another instance of instability and uncertainty. 

Perceived Lack of District Support 

Respondents in 4 of the 12 core subsample schools described the district as a primary threat to 
sustainability, often in strong, bitter terms (see Box 9.2). Staff at one school reported a lack of support 
from the district, particularly with regard to human capital management. As the principal explained, 

"The key challenge is. ..the district office— whether it is staffing, whether it is having quality people in our 
human resources ranks to be in the applicant pool.... The people that were in the pool, I wouldn't have 
hired to watch my dead dog." A member of the school improvement team also noted: 

We also have lost some people to other [district] schools. We've lost people that way. That kills 
the system.... It can be demoralizing at times to invest this amount of time into people and then 
to see that others in the district are able to just go in and [poach staff from us]. And people just 
leave. The system doesn't support the most needy schools. 

At another school, the lack of support was reported by nearly all respondents (including district staff) as 
one of the primary reasons for the school's history of low performance. At the conclusion of SIG, 
respondents expressed concern that the district could revert to their prior practices of sending the 
lowest-performing teachers to this school. Nearly every respondent, from parents to teachers to 
administrators, cited the district as the greatest challenge facing the school. 

Loss of Specific Interventions 

Respondents in 6 of the 12 core subsample schools pointed to specific interventions, resources, or 
programs that they would lose once SIG money was spent (see Box 9.4). Some of these specific items 
have implications for human capital. Specifically, respondents noted that they would not be able to 
continue professional learning opportunities at the current levels (three schools), extended day or 
afterschool programs would be cut (three schools), and technology purchases would be curtailed (two 
schools). The potential loss of the extended day at one school was particularly acute because it was 
perceived to be closely intertwined with other issues related to sustainability. If the extended day were 
cut, then teachers would lose the associated extra pay and would be less inclined to drive the long 
distance to the school. They would also lose a full class period, reducing the number of classes that this 
small high school could provide, including newer offerings such as Advanced Placement and music. 

Overall Prospects for Sustainability 

Thus far in this chapter we have presented a quantitative measure of sustainability using teacher survey 
responses, as well as a qualitative measure focused on risk factors for sustainability identified from site 
visit interviews and focus groups. We now combine these two measures to provide a broader summary 
of schools' projected prospects for sustaining improvements (see Box 9.5). 

Respondents from 2 of the 12 core subsample schools perceived their schools to have strong 
sustainability prospects. Not only did these schools receive the highest ratings on the teacher survey 
scale, but other interviewees expressed confidence that these schools had the resources to sustain and 
build upon their improvements. For example, respondents at one of these schools noted that, from the 
start, the school planned to use SIG to purchase materials and train staff to build capacity. An 
administrator said that the school applied for SIG "intentionally saying what is our plan for sustainability 
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and what can't we afford to do in Year 1, or 2, or 3 without some funds raised to support that." Thus, at 
the end of Year 3, the school was prepared for the absence of SIG. 


Box 9.5. Perceived Sustainability Prospects 

This analysis focuses on respondents' perceptions of their school's prospects for being able to sustain 
and build upon improvements. School classifications on sustainability prospects are based on Year 3 
interview and focus group data (see Exhibit B.30 for more detail on the analytic procedures). For this 
analysis, respondents refer to individuals from the following respondent groups: district 
administrators, principals, teachers, instructional coaches, external support providers, and school 
improvement team members. 

Perceived as having strong sustainability prospects 

• Qualitative data: No respondent mentioned any of the four risk factors to sustaining and 
continuing school improvement; and no respondent described the school's sustainability 
prospects in negative terms; AND 

• Survey data: Sustainability index scale average was at least 0.5 standard deviations (0.26) 
above the Year 3 scale mean (3.01). 

Perceived as having mixed sustainability prospects 

• Qualitative data: Respondents from at least three respondent groups reported on at least one 
of the four risk factors to sustaining and continuing school improvement, 3 but respondents' 
descriptions of sustainability prospects were mostly positive; AND 

• Survey data: Sustainability index scale average was within 0.5 standard deviations (0.26) of 
the scale mean (3.01). 

Perceived as having weak or low sustainability prospects 

• Qualitative data: Respondents from at least three respondent groups reported on at least one 
of the four risk factors to sustaining and continuing school improvement; 3 and respondents' 
described the school's sustainability prospects in mostly negative terms; AND 

• Survey data: Sustainability index scale average was below the scale mean (3.01) 

0 While theoretically possible, there were no instances in our data where respondents from only one or two respondent 
groups reported on one or more of the four risk factors to sustaining and continuing school improvement. 


At the other school, respondents were confident that "structures and systems" were sufficiently 
institutionalized that the practices they valued most would persist. As one school improvement team 
member explained: 

There's been enough key players in the school who have seen the changes and the 
metamorphosis that I think if [the current leadership team] were to leave this building and 
someone else were to come in, I think they would have the knowledge and wherewithal to say, 

"This is not how it's done" and "We do not think this will be effective." So I think there's enough 
people here— pockets in the various grade levels— to be able to see that and be able to move 
forward. 

In addition, this school's leadership had also worked with the district to ensure funding for 
"intervention" staff who had been paid through SIG. The school improvement team had identified 
"nonnegotiables" that the school could continue without funding from SIG, including weekly grade-level 
planning time. 
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Respondents from 6 of the 12 schools perceived their schools to have mixed sustainability prospects. 

In these schools, multiple site visit respondents described at least one potential factor in the coming 
year that could jeopardize their school's progress. However, these risk factors were balanced by 
reported changes to school culture that could sustain continuous improvement. For example, one 
teacher commented, "I don't think that [losing SIG funds] will hurt us much because we've built such a 
good foundation, so many sustainable things already," and a second teacher explained that staff and 
students now embrace the notion that "this is what we do here. This is the culture. We're successful." 
Still, multiple respondents (principal, teachers, instructional coach, and district official) all raised 
concerns about the potential loss of staff who had been funded through SIG. 

Another school had a challenging start to the SIG process, but by Year 2 it had put in place several 
efforts to build capacity, yielding one of the highest ratings for capacity-building efforts (see Chapter 8). 
At the conclusion of SIG, one teacher exclaimed, "The last two years have been amazing! ...I've been 
exhausted, they've gotten me back to feeling like this is what it's supposed to be, and this is what this 
grant was supposed to do." The principal noted: 

The other piece is the whole notion of capacity building. So hopefully— SIG wasn't a long-term 
fix, it's a short-term boost— and so hopefully we do have systems and structures in place that 
although the dollars are leaving, the knowledge and the know-how and the best practices and 
the research is there to continue the work around grade-level collaboration, around professional 
development. 

Respondents from 4 of the 12 schools perceived their schools to have weak sustainability prospects. 

For two of these schools, the survey sustainability measure was low relative to the other schools. In 
addition, respondents from multiple respondent groups at both schools expressed concerns about their 
school's leadership for the next school year and their leaders' ability to guide the school's continued 
improvement process. For example, at one of these two schools, the principal reported having 
responsibilities that took him outside the school, and expectations were that he was not going to return 
in the next school year. One teacher expressing uncertainty for the future said, "Even though I think he's 
done an okay job of building the vision, I just don't know that it'll stick if he goes." In the other school, 
respondents from multiple respondent groups, including the leadership team and teachers, expressed 
concern about the current school leadership. According to respondents, the school suffered from poor 
leadership during SIG and "a lot would fall into place with a solid leadership." 

At the other two schools with reportedly weak sustainability prospects, respondents expressed concern 
about losing staff and described their prospects for sustainability in distinctly negative terms. As one 
principal stated bluntly, "I just feel that some of this is going to go down the gutter." The principal at the 
other school explained that the district office was a notable barrier to improvement: 

The challenge has been to have us be a priority at the district level, and we truly are not.... The 
constant challenge is the district. It is. At the school, it's the mobility of my staff— having a fluid 
system of staff and having to continually rebuild. My AP [assistant principal] was pulled out last 
year to be a principal at another school.... There isn't a priority to turn us around. Our students 
and their families do not have a voice. And it hurts me to see that this district is taking 
advantage of them. 

The 12 core subsample schools' prospects for sustainability appear to be unrelated to several 
variables that might be hypothesized to predict sustainability, such as the relative size of SIG, SIG 
model, enrollment, or school level (see Exhibit 9.2). However, the two schools with stronger prospects 
of sustainability also were deemed to have a disruption from prior practice (see Le Floch et al., 2014). In 
addition, a school's predicted prospects for sustainability do generally appear to be related to the 
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percentage of teacher survey respondents who reported a positive change over the course of SIG. For 
example, Baltimore Bridge Elementary has both strong sustainability prospects and the highest 
percentage of teachers reporting improvement (97 percent). 


Exhibit 9.2. 

Core Subsample Schools' Sustainability Prospects and Other Variables 



Sustainability 

Prospects 

Average 

Enrollment 

2010-13 

Estimated 
Year 1 SIG Per- 
Pupil 

Expenditures 
as a 

Percentage of 
Overall Per- 
Pupil 

Expenditures 

Disruption 
From Prior 
Practice 

Percentage of 
Teachers 
Reporting 
Positive 
Change 

Sawbuck Elementary 

Weak 

350-550 

20% 

Yes 

0% 

Blizzard Bay Elementary 

Weak 

350-550 

7% 

No 

10% 

Proctor Point High 

Weak 

650-900 

13% 

No 

31% 

Coral High 

Weak 

1200+ 

6% 

No 

43% 

Paul Bunyan High 

Mixed 

1200+ 

16% 

No 

49% 

Elmsville High 

Mixed 

1200+ 

22% 

No 

53% 

McAlliston High 

Mixed 

650-900 

32% 

No 

57% 

Island Bay Elementary 

Mixed 

350-550 

120% 

No 

67% 

Gale Secondary 

Mixed 

650-900 

15% 

No 

76% 

Meribel High 

Mixed 

350-550 

49% 

Yes 

92% 

Sterling Slope Elementary 

Strong 

350-550 

6% 

Yes 

85% 

Baltimore Bridge Elementary 

Strong 

650-900 

7% 

Yes 

97% 


Source: SST respondent interviews and focus groups, spring 2013; SST teacher survey, spring 2013. 
Notes: Includes 12 core subsample schools. All school names are pseudonyms. 


Ownership of Reform 

Researchers have argued that school improvement actions are most likely to be sustained when they are 
internalized— that is, when they are no longer driven by an entity external to the school but are 
understood, embraced, and self-generative (McLaughlin & Mitra, 2001; Stokes et al., 1997). Coburn 
(2003) describes this shift in ownership through which both the authority for and knowledge of the 
improvement process transfers to school actors from an external source. Related to the notion of buy- 
in, ownership is a deeper concept, through which stakeholders have become personally invested in the 
improvement actions. Ownership of reform comes about when they are able to "sustain, spread and 
deepen reform principles" (Coburn, 2003, p. 7). Thus, we hypothesize that evidence of a shift in 
ownership would be a potentially strong predictor of sustainability after SIG ends. Flowever, as the 
following school profiles demonstrate, ownership and engagement of the school staff in the reforms and 
the improvement process may not be enough to ensure sustainability. 

Respondents at 2 of the 12 core subsample schools provided evidence of ownership of the school 
improvement process by Year 3 of SIG (see Box 9.6). In the remaining 10 schools, not enough data were 
collected to make an assessment of ownership. At one of these schools, implementing the restart 
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model, the school's charter leadership intentionally designed its governance plan to give teachers more 
responsibility over school decisions as the grant period progressed. The principal explained that the 
school had a three-year model of "my school, your school, our school." As he recounted: 

Year 1 was about me being the boss here, from my perspective, setting [the] tone for everything, 
everyone. Year 2 was about me releasing that to my leadership team, making sure they were 
being the boss of their fiefdoms. Year 3 is about teachers being the boss. We had t-shirts made, 
and [we] talk about this all the time: my school, your school, our school. 


Box 9.6. Perceptions of Ownership of Reform Efforts 

This analysis focuses on whether respondents perceived the reform efforts being implemented as 
within their control. Classifications on ownership are based on Year 2 and 3 interview and focus group 
data (see Exhibit B.31 for more detail on the analytic procedures). 

Evidence of Ownership 

• At least one administrator (district administrator, principal, assistant principal) and at least 
two nonadministrative staff (teacher, instructional coach, parent) described teachers, 
administrators, or other stakeholder groups as contributing ideas, adjusting improvement 
strategies, taking on new responsibilities, and/or articulating the rationale for change. 


The CMO representative explained that in the second year, the principal focused with his leadership 
team on how to shift the school's emphasis from "this is [the principal's] school" to "this is 700-and- 
some-odd families' school, this is all the teachers' school, and everybody owns what happens here." The 
CMO representative further explained: 

[Leadership] talked with the teachers, talked with staff, like how can we get to a place where we 
really own it. So they [the leadership] shifted the lens, whether it's staff meetings or it's 
community meetings for kids. And they're really pushing ownership and voice to teachers and 
staff, which has been really exciting.... So we're actually seeing a lot of site-level innovations 
happening because [the principal] is embracing taking feedback from teachers and saying, "If 
they have a great idea, I'm going to try it." 

In Year 3, this school piloted a kindergarten program originally proposed by teachers. In addition, 
teachers explained that they take part in monthly steering committees, where staff members discuss 
plans for the future and their ideas about it with teachers and administrators. These meetings are open 
to any staff member and focus on a variety of topics (e.g., math, reading, culture, incentives, trips, 
behavior management). "The leadership team is very open," a teacher explained, adding that "it's about 
teacher feedback and what we can do to make the school better." 

At the other school, stakeholders were reportedly disengaged and apathetic prior to SIG. According to 
accounts from multiple respondents, the teachers, parents, and students believed that their school was 
not on par with "real" high schools, and they had reportedly become resigned to second-tier status. 
However, the principal who was new to the school in the second year of SIG took on the challenge of 
encouraging the school community— starting with the teachers— to offer solutions, to embrace reform, 
and to not settle for second best. He admitted this required a change in leadership style, trusting and 
empowering staff to enact their own ideas. As he related: 

The type of leadership that it requires to run this school is very different than what I would have 
expected that I would have to bring in. You don't want to have your hands on every aspect of 
what happens on your campus. What you end up doing is being present in every aspect, but you 
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also have to make a conscious decision to let the opinions be heard, and whether you have a 
slightly different opinion or not, you have to look at what's best for your stakeholders.... In order 
for this to really move forward, you kind of have to trust the process. That requires a little bit 
more patience. 

He described his interactions with teachers in the school: "I sort of feed them the idea, encourage them, 
feed them the idea, encourage them, and then once it became theirs, then we're ready to move 
forward." Likewise, he encouraged students to approach him with their own ideas for the school, 
whether for a new club, afterschool event, or other resources. And when students had suggestions, "I 
always put it back on them: 'Give me the people that are going to be involved, what your expectations 
are, and then I'll take it back to the district office.' ...I love the fact that the kids now expect that we 
should have the same resources and activities as other schools." 

By the end of Year 3, multiple respondents— including the principal, assistant principal, coaches, and 
teachers— explained how the entire school community was more engaged, confident, and reliant on 
each other. By way of some examples, one teacher noted, "I think one of the things that changes the 
way we are looking now is before we used to get outside experts to present to us, and now we're trying 
to be our own experts." And the assistant principal observed, "I think that's one of the things that I've 
seen in the teachers. I see them now taking ownership of that and doing the research and figuring things 
out. This year we're building a master schedule, and these teachers are on it." Finally, one teacher 
summed up the shift in ownership as follows: 

A lot of the change is people driven. It's a slow change, but it comes way more solid when it's 
built in relationships. I'll give you the example of the [classroom] walkthroughs. It started as 
basically an administrative leadership team. It morphed through conversations through risk- 
taking to teachers saying, "We’re comfortable with that." ...It just keeps evolving, and it's a 
natural evolution. How will this best help us improve? I imagine it will continue morphing to 
whatever we need at that point.... Our next steps are to increase accountability. And we're at a 
point because of the relationships where we can ask accountability from each other without 
being offended. 

However, in spring 2013, respondents, including school leadership and teachers, also pointed to several 
factors that could threaten the school's ability to maintain progress and continue to improve. Thus, 
although this school provided evidence of a shift in ownership, the presence of these risk factors 
ultimately placed it in the mixed sustainability prospects category. A key risk factor was the expectation 
that the principal would not be returning in fall 2013. One teacher said that she could not speak to the 
school's prospects "because we don't know who our principal is going to be" and "everything changes 
when there's a new principal." Although the school had managed to negotiate with the district to retain 
some SIG-related activities, multiple respondents expressed concern that the district would neglect the 
school's needs— as had reportedly been the case in the years prior to SIG. Indeed, respondents 
(including the principal, external support provider, teachers, parents, and district administrators) 
described their school as an "orphan," in part because of its geographic distance from the district office. 
Because of this history, staff were fearful that it would be too easy to slip into the previous dynamic with 
the district office. As one teacher explained, "When you ask me what it's going to be like in five years... I 
think it could be back to where it was. Because if somebody doesn't get up there in the district and 
realize the history, then I think we might be doomed to repeat history." And the school's external 
support provider summed it up this way: 

I have been really irked by the lack of district support from the get-go.... I just don't think there 
was enough effort put into this by the district office, and I'm still not really pleased with the kind 
of support we're getting.... I'm enjoying watching the school improve, [and] we have some data 
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that is very good. We had the biggest gain of any high school in our district. We had one of the 
best high school exit exam improvement rates in the county. We are making progress. I just don't 
want it to fall flat on its face when all the [money] goes away. 

Thus, although this school built capacity and made some notable improvements under SIG, these 
improvements may prove fragile given the potential change in leadership and reportedly precarious 
relationship with the district. 

Chapter Summary 

Although SIG constituted an important opportunity for schools, it was always envisioned to be a 
temporary infusion of funds that would hopefully build capacity that could then help sustain 
improvement after the grant period ended. As we saw in Part II of this report, our 25 core sample 
schools did report making efforts to build capacity related to human capital in the first two years of SIG. 
As reported in the prior chapter, among our 12 core subsample schools, those that made the most 
vigorous efforts to build human capital capacity also appeared to have the greatest positive changes in 
organizational capacity. And as reported in this chapter, core subsample schools that had higher 
organizational capacity in Year 3 of SIG also tended to have higher sustainability prospects based on 
teacher ratings. However, the sustainability of any improvements may prove fragile. Even schools that 
appeared to build capacity and experience a shift in ownership of reform also appear to be threatened 
with continuing staff turnover, change in leadership, loss of specific interventions, or an unsupportive 
district context. Looking across the 12 core subsample schools, the overall prospects for sustainability 
seem strong in 2 of them but mixed or weak in the other 10. Thus, although our data suggest that the 
schools in this subsample experienced some change during the course of SIG, lasting improvements may 
remain elusive. 
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Report Conclusions 


In the decades since U.S. schools have been tracking student achievement with assessments that enable 
comparisons across schools and student subgroups, divergent performance levels have been apparent 
and troubling. Federal policy has sought to stimulate and sustain improvement among the lowest- 
performing schools, through successive reauthorizations of ESEA (i.e., the Improving America's Schools 
Act and No Child Left Behind), and through programs specifically focused on fostering improvement 
models— such as the Comprehensive School Reform Demonstration Program and the SIG program. SIG 
in particular was intended to stimulate dramatic change in low-performing schools through personnel 
changes, professional learning opportunities, extended learning opportunities, and use of data to inform 
instructional decisions— all facilitated through an unprecedented infusion of federal funds. 

The Study of School Turnaround was designed to examine how a variety of SIG schools approached the 
improvement process over the three-year grant period, and how SIG funds and strategies contributed to 
this process. In doing so, we were able to document schools' experiences in the initial phases of SIG, 
perceptions of performance problems, sets of improvement actions being adopted and implemented to 
drive school change, efforts to build human capital, changes in overall organizational capacity, 
perceptions of school improvement, and anticipation of sustaining an improvement process. In addition, 
as we developed familiarity with school stakeholders over time, we became more cognizant of each 
school's context, history, challenges, and potential for longer-term success. 

In previous chapters of this report, our findings were organized by topic, with a focus on building human 
capital, organizational capacity, and sustaining the improvement process. In this conclusion, we 
synthesize our overarching findings about school change. 

Most of our case study schools appeared to make positive changes to their organizational capacity, 
and most respondents in these schools reported improvement over the course of SIG. We are not able 
to determine whether these apparent changes were because of SIG, nor are we able to verify that these 
apparent changes also led to changes in actual student achievement. Flowever, we can say that most 
respondents in most of our schools perceived that their schools had made positive changes over the 
course of SIG, most often with regard to school climate. Respondents also described improved 
instruction, data use, and collaboration. In several schools, respondents described their improvements 
in strong language, for example by explaining that prior to SIG, “There was really nothing going on in 
classrooms. Kids were running in the hallways [and] using profanity. It just wasn't a school. Now we are 
a school. It's a school where you see learning. It's a school where there's a feeling of safety.... It went 
from a warehouse of children to a school." Thus, although we cannot conclude that the various SIG 
interventions and supports precipitated the positive changes described in these schools, most 
respondents concluded that their schools were better off at the end of SIG, at least in some respects. 

Nevertheless, some schools did not appear to change much under SIG. For example, respondents in 
one school described SIG as a "business-as-usual" approach, and this school showed no demonstrable 
change in organizational capacity over the course of SIG. 

The schools that had the largest reported changes in organizational capacity by Year 3 of SIG were 
those that started at the lowest capacity levels in Year 1. Moreover, these are also the schools that 
reported making numerous efforts to build human capital. As reflected in SIG requirements, and 
consistent with prior studies, a minimum threshold of capacity may be necessary to enable the process 
of school change. Flowever, this finding suggests that it may be possible, with some concerted effort, for 
low-capacity schools to build organizational capacity. In addition, we found that the schools with higher 
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initial capacity were able to maintain or even improve their organizational capacity over the course of 
SIG, consistent with a hypothesis that capacity could beget capacity. 

Leadership appears to matter. This is a recurrent finding in studies of school improvement, and our 
study was no exception. Among our sample, schools in which respondents described improvements 
were more likely to be led by principals that reportedly demonstrated elements of strategic leadership, 
including having a theory of action for making improvements in their school. Furthermore, schools in 
which higher percentages of teacher survey respondents reported improvement had the highest-rated 
principals with regard to transformational leadership (principals who can motivate and engage their 
staff) and instructional leadership (principals who are knowledgeable about instructional issues). As 
important as strong leaders appear to be, identifying, recruiting, and retaining the most skilled principals 
in chronically low-performing schools is a persistent challenge. Mandating principal replacement in SIG 
schools appears to have in many cases brought stronger leaders to schools that needed a change in 
leadership. At the same time, we observed a continued pattern of frequent leadership turnover in 
several of our case study schools— a pattern which reportedly undercut the school's progress. When 
school leaders depart— particularly strong, well-regarded ones that are credited with improvements— 
the sustainability of school improvement efforts may be threatened. 

Districts may be a notable asset in the improvement process, but some districts do not appear to work 
in the interest of their struggling schools. Our case study sample included districts that reportedly 
established structures, data platforms, meeting times, and other supports in service of their SIG (and 
other low-performing) schools. In some of our case study schools nested within these districts, 
respondents described their district supports favorably, even stating that continued district support 
would enable them to sustain improvements after SIG. In contrast, some schools expressed concern 
about the lack of district support, and described district actions that could jeopardize their capacity- 
building efforts. These schools described district practices that displaced their strongest teachers and 
leaders. As much as districts can do to support the change process in low-performing schools, our study 
provides examples of how districts may inhibit improvement at the school level. 

School change appears to be fragile. Of the 12 schools in our core subsample, only 2 appeared to be in a 
strong position to sustain an improvement process. Six others may have the capacity to weather the loss 
of SIG resources, but they also reportedly had some notable risk factors, and despite reported 
improvements over the three-year grant period, the newly-built capacity could prove fragile. Among the 
risk factors described by respondents, nearly all were associated with human capital, particularly the 
anticipated loss of teachers, principals, or other key staff. District policies, especially those related to the 
placement and retention of teachers appeared to underlie some of these risk factors. Schools that 
appeared to have stronger prospects for sustainability were also those in which teacher survey 
respondents reported that the school had changed in positive ways, and were more likely to have 
experienced a disruption from prior practice at the beginning of SIG. 

Our study suggests several challenges for education stakeholders seeking to enact programs and policies 
designed to break the cycle of persistently low-performing schools: 

• Flow can programs and policies anticipate and minimize unintended consequences? Policies that 
mandate teacher replacement, for example, may inject new energy into a school with a 
dysfunctional staff culture. But district policies and standard operating procedures can 
undermine the intent of this policy, resulting in the placement of less effective teachers in low- 
performing schools (colloquially described as the "lemon dance"). 

• Flow can programs and policies better take into account variation in school context and history? 
Low-performing schools are not all the same, yet policies that offer limited options make 
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implicit assumptions about the characteristics of schools, their contexts, and where they are in 
the improvement process. For example, although some schools in our sample appeared to need 
a dramatic jump-start for their improvement efforts, others had already begun reform efforts— 
some quite extensive— prior to receiving their grants. Schools like these appeared to need less 
of a "jolt"— such as one that accompanies the replacement of a principal— and more continuity 
to be able to consolidate and build on a solid foundation. 

Such policy challenges could be better addressed with more nuanced knowledge about how different 
approaches to reform play out in different circumstances. Under what conditions do prescriptive policies 
foster change? Under what conditions is it more effective to build incrementally on a solid foundation? 
What types of district supports set the conditions for turning around low-performing schools, and what 
district policies seem to get in the way? And finally, do requirements like those in SIG help schools 
continue to build the capacity that they really need to improve and sustain improvements? 

The school change process is complex and crafting policy that acknowledges this complexity while 
compelling change has challenged policymakers for decades. Low-performing schools are not blank 
slates, on which new interventions and individuals can be imposed and assumed to stimulate better 
outcomes for students. These new policies are inserted into a complex policy context, history, and set of 
assumptions about each school. Still, our study provides evidence that these chronically low-performing 
schools can change in some respects, at least in the short term, with a great many efforts to build 
human capital. To sustain these changes, however, it appears that an equally great effort may be 
needed to retain any hard-won improvements. 
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Appendix A. Study of School Turnaround Codebook, 2010-13 

Code Correspondence by Year of Data Collection 

The following tables present the code names, definitions, and examples used to describe codes in the SST codebook. The last three columns of 
the tables illustrate changes to the codes across the study's three years of data collection. Check marks in those columns identify the data 
collection year(s) in which a particular code was used. For years in which the code was not used, the columns identify which, if any, codes were 
used that year that would have been applied to equivalent segments of data. If none of the codes from that year applied to equivalent segments 
of data, the column indicates that no equivalent code was used that year. 


PARTICIPANT BACKGROUND (#): Codes in this category apply to segments of data related to the background of interview and focus group 
participants. Apply these codes to segments of data that outline individual participants' histories at the school or district. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

#participant 

background 

Information on the participant's 
background in education. 

Prior jobs, prior districts, 
experience with similar 
schools/school turnaround, 
professional degrees 

#long history at school 

#new to district 

#new to role 

Y 

Y 

#new to 

school 

The participant was new to the 

school as of the 2010-11 school 

year or later. This code is meant 
to be used to distinguish staff 
that joined the school post-SIG 

versus staff that were there 

pre-SIG. In Year 1, this code also 
included participants that 
started working at the school in 
the 2009-10 school year. 

Teacher, principal, external 
provider 

Y 

Y 

Y 
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SCHOOL CHANGE (SC_): Codes in this category apply to segments of data related to the school change process. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

SC_Challenge 

Challenges or "areas for improvement" 

Reading or math instruction. 

““Challenge 

SC_Problem 



that are identified by study participants 

teacher morale, serving specific 

““Problem 

Definition 



as being new or on-going sources of 

student populations more 

definition 




difficulty. For Year 3, this code also 

effectively (ELL or special ED), 





includes descriptions formerly coded as 

parent/community involvement, 





"problem definition," or conditions 

lack of financial resources. 





identified by school stakeholders as 

ineffective teachers, poor 





preventing the school from moving 

leadership, violence or behavior 





forward with improvement efforts. 
Challenges may be specific to Year 3, or 
they may be consistent over time. 

issues 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

SC_Coherence 

Data that describe the degree to which 
the policies of a school reflect consistent 
goals; the strategies employed are 
clearly designed to foster achievement of 
these goals; and the barriers and 
detractors from the goals and strategies 
are systematically removed. This code 

can describe evidence of coherence as 

well as a lack thereof. It can also be used 

to capture system coherence between 

the district and the SIG school under 

study. 

District-wide and school-wide 

implementation (across grades, 
content areas); if reform 
strategies address the school 
challenges; the need to focus 
because there are too many 
reform strategies; alignment 

between assessments and 

teaching standards; alignment 
among levels of organization 
(such as school, district, and 
state); alignment between 
reform strategies and goals; 
districts taking lessons learned 
and practices from schools 
receiving SIG and applying them 
to other schools in the district; 
consistency of reform strategy 

over time 

%Coherence 

SC_Alignment 

of Practices 

y 

SC_Culture_clim 

ate 

Data that refer to changes in the culture 
and climate (i.e., feeling) at the school 
site. Covers the entire spectrum of 
change from "no change" to "drastic 
change." Can be the perspective of the 
study participant or based on the coder's 
analysis of the text. Not to be confused 
with buy-in or morale. Will often be 

double-coded with codes in the School 

Climate ( A ) section. 

Climate is happier; teacher and 
student body feeling more 
settled; school feels like it has 
more energy; climate is better, 
improved culture of 
collaboration, student behavior 

No equivalent 

code 

SC_Perceived 

Change 

y 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

SC_District Level 

Despite being included in the "school 
change" category, this code is meant to 
capture data on district-level changes, 

either in Year 3 of SIG or over the course 

of the three years of the grant. If district 
staff are describing changes in support to 
SIG schools, make sure to double-code 

with SS District. Includes the rationale 

for district-level changes being discussed. 

District restructuring, changes in 
district leadership, changes in 
district goals and priorities, 
changes in district strategies 

No equivalent 

code 

SC_Perceived 

Change 


SC_Goals 

Data that address district or school goals 
either past, present, or future. The 

extent to which various school 

stakeholders are aware of school or 

district-wide goals. Descriptions of how 
goals have been 

implemented/operationalized either in 

Year 3 or over time. The rationale about 

which goals to adopt, amend, or 

abandon. This code will also be used to 

capture changes in school or district 
goals over the course of SIG (Year 1 to 

Year 3). This code can also be used to 
capture data on future goals that will 
post-date SIG. Includes rationale for 
changes in goals. 

Improving test scores, lowering 
dropout rates, higher attendance 
rates, more teacher collaboration 
and col legiality, a more defined 
discipline process; changing focus 
from one goal to another and 
rationale for this choice; 
descriptions of future goals for 
the school once SIG is over, like 
continuing to improve graduation 

rates or test scores 

* Priorities 


V 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

SC_Monitoring 

Change 

Data that address whether and how the 

school, district, or state is monitoring the 
turnaround process in SIG schools. 
Different than the SIG_Processes code 
because this is monitoring school change 
more broadly and not specifically 
evidence of SIG compliance and 
monitoring activities. Includes rationale 
for specific monitoring processes or 
practices. 

Includes analysis of benchmarks, 

school and classroom walk 

throughs, and leadership 
meetings held to assess progress 
toward goals 

No equivalent 

code 

Y 

Y 

SC_Other 

Any data (other than climate/culture, 
student outcomes, perceived strategies) 
that reference perceived change at the 
school. Covers the entire spectrum of 
change from "no change" to "drastic 
change." Can be the perspective of the 
study participant or based on the coder's 
analysis of the text. 

More focused school goals, no 
change in support from principal 

or district leaders 

No equivalent 

code 

SC_Perceived 

Change 

Y 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

SC_Perceptions 

Data that capture district and school- 

Reflections on setting school 

No equivalent 

SC_Perceived 


on change 

level reflections on aspects of school 

goals and establishing a vision; 

code 

Change 


process 

change either in Year 3 of SIG or over 

building teacher support; 





time. Reflections could address pace of 

working with your school 





change, progress towards school or 

community and/or teacher's 





district goals, challenges associated with 

union; implementing new 





school change or the implementation of 

curricula, instructional strategies. 





change strategies, or "lessons learned" 

or technology use, etc.; role of 





about turning around low-performing 

strong leaders setting realistic 





schools. Captures data on the change 
process in SIG schools that is more 

holistic and not broken down into 

specific components of change. 

expectations for change 





A-6 


Case Studies of Schools Receiving School Improvement Grants: Final Report 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

SC_Strategies 

Data that refer to changes in school 
strategies at the school site. Does not 

need to be related to SIG. Could be 

related to implementation of activities, 
the addition or subtraction of programs 
and staff. Refers to deepening of 
practice— descriptions of activities or 
programs that the school, teacher, 
district began working on in prior years 
and are continuing to work on this year. 
Covers the entire spectrum of change 
from "no change" to "drastic change." 

Can be the perspective of the study 
participant or based on the coder's 
analysis of the text. Includes the 
rationale for changes in strategies. Will 
often be double-coded with codes in the 
Domains of Activity ($) section, 

SS_External Provider, SS_lnstructional 

Coach. 

More teacher collaboration, 
adding or subtracting school 
positions or a parent or 
community liaison, adding an 
afterschool program, change in 
curriculum, added coach support, 
improved teaching practices, 
teacher replacement, leadership 
changes 

No equivalent 

code 

SC_Perceived 

Change 

Y 

SC_Student_out 

comes 

Data that refer to changes in student 

outcomes. Can be at the school and 

district levels. Covers the entire 

spectrum of change from "no change" to 
"drastic change." Can be the perspective 
of the study participant or based on the 
coder's analysis of the text. Includes the 
rationale for changes to student 

outcomes. Will sometimes be double- 

coded with A Attendance. 

Changes in school status (such as 
going from an "F" school to an 
"A" school); making AYP; 
improvements on reading and 
math assessments; improved 
attendance and graduation rates 

No equivalent 

code 

SC_Perceived 

Change 

Y 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

SC_Vision 

Discussion of long-range vision for the 
school. What principals, teachers, 
parents, students, or district personnel 

would like to see the school become over 

time. Includes the rationale for school 

vision. 

All students proficient, life-long 
learners, college-goers, school as 
community resource, etc. 

C_mission 
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SIG PROCESSES (SIG): Codes in this category apply to segments of data that describe how SIG schools are navigating the SIG process. These 
codes can describe aspects of SIG at any level of the system (state, district, school) and should be paired with the appropriate multipurpose code 
to separate district-level from school-level data. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

SIG_Processes 

Information on processes related to 
the administration of SIG, including the 
application and grant revisions 
process, model selection, grant 
restrictions, distribution of funds, how 
SIG activities are implemented, SIG 
oversight, and support for SIG 
implementation. This includes school- 

and district-level data. 

When schools received Year 3 

funding, whether they needed to 
carry-over Year 2 funding, how their 
grant activities are monitored by the 
district (if at all), whether schools 
can apply for funding extensions 

post-grant 

SIG_Application process 
SIG_Distribution of funds 
SIG_Funding restrictions 
SIG_Grant revisions 
SIG_Model selection 
SIG_Oversight 

SIG_Support 

SIGJmplementation 

SIG_Planning 

SIGMisc 

Y 

Y 

SIG_Funded 

Activities 

Data that identify specific activities as 
being SIG-funded. Can be new 

activities for Year 3 or also discussions 

about SIG-funded activities from Year 

1 or Year 2. Will probably be used in 
conjunction with other codes from the 
"Domains" category. Data can be 

coded SIG-funded if the coder knows it 

is SIG-funded even though the 
respondent does not explicitly say so. 

PD, curriculum, staff positions 
(coaches, counselors), technology 

*SIG-funded 

Y 

Y 

SIG_Perceptio 

ns 

Data that capture stakeholder 
perceptions about the SIG grant. These 
perceptions may be positive or 
negative; they may be process 
oriented, implementation oriented, or 

outcomes oriented. Can also be used if 

a study participant is not aware of SIG. 

How the community thinks of the 
usefulness of the grant, how 
students feel the grant has changed 
the school, if teachers, parents, or 
students are aware of the grant 

Y 

Y 

Y 
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DOMAINS OF ACTIVITY ($): Codes in this category apply to segments of data that describe specific strategies, tactics, or activities that schools 
are continuing to use or have begun using during the 2010-11 school year. These strategies can be funded through SIG or through other funding 
streams. The codes are aligned with the study's conceptual framework. The codes are intended to be broad, so they should include all data that 
touch on the topics identified by the code. Also, the multipurpose codes (especially change strategy) will be used in conjunction with these 
domain codes to identify specific stakeholder groups or system-level actors involved with these activities. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

$Curriculum_l 

nstruction 

Data about curriculum and/or instructional practices. 
Includes existing or new curriculum, as well as 
modifications/enhancement of existing curriculum. 
Includes aligning new or existing curriculum to 
district, state, or national learning standards. Also 
includes discussions of schoolwide efforts to improve 
instructional practices. Includes issues related to 
lesson planning, how students are grouped for 
instruction, and how students are placed into 

courses. 

New or current math, ELA, or 
ELL curriculum; new 
standards; guided reading 
strategies 

$Curriculum 

$lnstruction 

S 

S 

$Common 

Core 

Data related to planning for and implementing the 

Common Core State Standards. Could be related to 

program coherence, adaptation of existing 
instructional or curricular strategies, assessments, or 
professional learning. Use in conjunction with other 
domain codes when necessary. 

Implementation, planning, 
program coherence, 
professional learning, 
curricular strategies, 

instruction 

No 

equivalent 

code 

No equivalent 

code 


$Data Type 

Data that reference type of data being collected or 
analyzed. 

State assessments, district 
assessments, school 
assessments, student work 

V 

No equivalent 

code 

No equivalent 

code 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

$Data Use 

Data that reference how data are being used by 
administrators, staff, and external stakeholders. 
Communication and interpretation of data, level of 
data literacy; frequency of data use to drive decisions 
and modify instruction. 

Data used to evaluate student 

performance, data to 
differentiate instruction, 
timing of data collection, 
process for sharing data with 

teachers 


V 

V 

$Evaluation 

Data that reference activities related to school or 

district evaluation systems. Includes new and existing 
evaluation systems, as well as changes to existing 

systems. 

New review practices for 
principals, teachers, schools; 
different people involved in 
review, different measures of 

success or failure 

$Evaluation 

Systems 

V 

V 

$lncentives 

Data that reference programs that create incentives 
for achieving desired outcomes. Can include existing 
incentive programs. Can also include other strategies 
to recruit and/or retain staff such as a "bonus" for 
hard to staff schools. Double-code with multipurpose 
codes to specify whether incentives are aimed at 
teachers, students, or administrators, and whether 
they are part of SIG or not. 

Monetary bonuses for 
teachers or principals linked 
to student test scores; 
incentives for teaching at 
hard to staff schools; rewards 
for students for good 
behavior, attendance, etc. 

✓ 

S 

S 

$Professional 

Learning 

Data that reference any type of professional learning 
activities offered by the state, district, or school. May 
also be associated with building capacity. In such 
cases double-code with the capacity multipurpose 
code. Includes frequency of professional learning, 
who provides professional learning, who attends 
professional learning, perceived usefulness of 
professional learning, support for implementation of 
professional learning, and content of professional 
learning. 

Professional learning series or 
one-time events, off-site 
training or on-site training, 
workshops, structured 
classroom observations, 

conferences 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

$Staffing 

Data that reference staffing decisions or conditions. 

Can be at any level— state, district, school. Includes 
changes in administrators, teaching staff, classified 
staff, teaching assistants, substitutes, instructional 
coaches, etc. Can include addition of new staff 
and/or changing roles and responsibilities. Also 
includes where staff were hired from if they were 
added to the teaching staff (e.g., from district layoff 
list or outside of district). Double-code with 
multipurpose codes to add detail. 

Replacing existing teachers 
with different ones (this could 

have occurred because of SIG 

model, or it can be something 
that is planned); hiring 
custodians, cafeteria staff, 
office staff; adding coaching 
positions; changes/additions 
to principals or other 

administrative staff 

Y 

Y 

Y 

$Technology 

Data that reference technology use. Can describe 
technology that has been purchased or distributed as 
part of a school's improvement efforts, perspectives 
on technology use within school or district, overall 
strategies related to technology use. Double-code 
with descriptive codes when applicable. 

New computers, smart boards 
for classrooms, laptops for 
teachers, overall strategies 
related to technology use, 
perspectives on technology 
use within school or district, 

new software 

Y 

Y 

Y 

$Use of time 

Data that reference strategies for extending, 
shortening, or restructuring learning time for 
students, strategies for restructuring work time for 
staff, amount of time being added, subtracted or 
restructured, and the stated rationales for their use. 
Includes academic afterschool activities like tutoring 

programs. 

Saturday school, academic 
afterschool activities, block 
scheduling, extra class periods 

Y 

Y 

Y 
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Leadership and Staff Capacity (!): Codes in this category apply to segments of data that describe the capacity and leadership roles of different 
types of staff at the school. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

IGovernance 

Data that address how decisions 

Who the principal asks for input 

$Governance 




get made at the school or 

when making decisions; how 





district. Whether decision- 

much input teachers get into 





making is a formal or informal 

the decisions about their roles 





process. How the school or 

and responsibilities; to what 





district is structured for 

extent parents have a role in 





leadership and oversight. 

decision-making; description of 





Descriptions of positions like 

leadership team; extent to 





leadership team, school site 

which the school (rather than 





council, leadership support 

the district, state) have 





staff, teacher leadership 

authority to make decisions 





positions. Perceptions of the 

effectiveness of school or 

district governance structures. 

Also includes issues related to 

the school's level of site-based 

autonomy in decision-making 
and/or flexibility with regard to 
district/state directives. 

about practice 




iOther school 

Data that describe how other 

Assistant principals or coaches 

*School leader 



leadership 

school leaders perform their 
duties at the school. Their styles 
of leadership; the amount of 
authority they have to make 
decisions; support for other 

leaders at the school site. 


*lnstructional coach 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

IPrincipal 

Leadership 

Information related to aspects 
of the school principal's 
leadership, including style, 
changes to leadership, 
effectiveness of leadership, and 
perception of leadership. This 
code will help define the extent 
to which principals are strategic, 
transformational, and 
distributive leaders. In coding, 
please review the definitions of 
each of these styles so that you 
are attuned to evidence specific 

to them. 

A new principal, authoritarian 
leadership, distributive 
leadership, good or bad 
perceptions of the principal's 
leadership style and/or ability 

$Leadership 

V 

V 

IStaff 

Capacity 

Data that address the extent to 

which different staff members 

at the school or district 

(teacher, coach, principal, AP, 
district) have the capacity to 
carry out their respective roles 

at the school. Could be evidence 

of high capacity, low capacity, 

or somewhere in between. 

Poor teaching skills, lack of 

instructional materials for 

teachers, excellent managerial 
skills; resources (e.g., time, 
materials) for planning and/or 
providing instruction 

*Capacity + stakeholder type 
(Teacher, *lnstructional coach, 
etc.) 



ITeacher 

Leadership 

Data that describe the 

leadership opportunities for 

teachers at the school and how 

teachers perform in these roles. 

Teacher leaders, PLC leads 

$Leadership 

Teacher 
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SCHOOL SUPPORTS (SS): Codes in this category apply to segments of data that describe supports provided to the school. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

SS_District 

Data that describe the district 

role in supporting school 
improvement or school 

turnaround. How district office 

is involved in the school. 

Perceptions of 
usefulness/effectiveness. If 
descriptions also involve 
changes from previous years, 
double-code with perceived 
change. 

Support for principals, PD, 
mandating policies or practices, 
policies, practices, or decisions 
made during SIG that hinder or 
help the change process more 
broadly 

* District 



SS_External 

Provider 

Data that describe the external 

provider role in supporting 
school improvement or school 
turnaround. How providers are 

involved in the school. 

Perceptions of 
usefulness/effectiveness. If 
descriptions also involve 
changes from previous years, 
double-code with perceived 
change. 

PD, instructional coaching, 
school oversight and 
management (EMO/CMO) 

activities 

*External support provider 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

SSJnstructio 

nal Coach 

Data that describe the 

instructional coach role in 

supporting school improvement 

or school turnaround. How 

coaches are involved in the 

school. Perceptions of 
usefulness/effectiveness. If 
descriptions also involve 
changes from previous years, 
double-code with perceived 
change. 

PD, working directly with staff 
to improve instruction, 
leadership decisions 

*lnstructional coach 

V 

V 

SS_State 

Data that describe how the 

state supports the school or 
district in their improvement 

efforts or the state's role more 

generally. Perceptions of 
usefulness/effectiveness. 

PD, support for SIG grant 
revisions, funding 

*State 

V 

V 

SS_Union 

Data that describe the union's 

role in the school or district. 

This could be a supportive role, 

or one that is more in conflict 

with the school's improvement 
plan. Union perspective on SIG 

or SIG-related activities should 

be double-coded with the 

appropriate descriptive code 
(e.g.. Evaluation, Staffing, SIG 
Processes). 

Contract negotiations, support 
or lack of support for SIG 

*Union 
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CONTEXT (C): Codes in this category apply to segments of data that reference aspects of school context including, but not limited to, past or 
present reform efforts, school history, academic performance, and student demographics. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

C_Academic 

performance 

Data that reference organization-level 

comments about the academic 

performance at the school or district 
generally. Can be this academic year 
or historical (e.g., history of low 
performance). 

Low academic performance, changes 
in academic performance levels over 
time, patterns of performance among 
different groups of students, 
achievement gaps, reasons for high or 
low levels of academic performance 


No equivalent 

code 

No equivalent 

code 

CCommitme 

nt to school 

Data that reference the extent to 

which respondents express 

commitment to the school 

themselves or describe perceptions of 
other people's commitment to the 
school. Double-code with parent, 
teacher, student, etc. depending on 
the stakeholder group that is being 

described in the data. 

Longevity at the school, level of 
involvement in school activities, both 
professional and extracurricular 

V 

No equivalent 

code 

No equivalent 

code 

CCommitme 

nt to students 

Data that reference the extent to 

which respondents express 

commitment to the students 

themselves or describe perceptions of 
other people's commitment to the 
students. Double-code with parent, 
teacher, principal, etc. depending on 
the stakeholder group that is being 

described in the data. 

Descriptions of time spent with 
students that goes beyond traditional 
job expectations, participation in 

extracurricular activities as a mentor 

or a coach 

V 

No equivalent 

code 

No equivalent 

code 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

C_Community 

relations 

Data that reference the relationship 
among community members. This 
includes the relationships between 
community members, community 
members and students, community 
members and the school/school staff. 

The relationship (good or bad) 
between community members and 
the principal and/or school. The 
relationship between two separate 
groups or different neighborhoods 
within the same community. 

V 

No equivalent 

code 

No equivalent 

code 

C_Culture 

Data that reference the type of 

culture that exists at the school or 

district and whether the school 

culture has changed over time and 
why. 

Positive, negative, supportive, 
collaborative, welcoming 

V 

No equivalent 

code 

No equivalent 

code 

C_District_sch 
ool history 

Data that describe events pre-dating 

SIG that were integral in shaping the 

current district or school context. 

Prior reform efforts; demographic 
shifts; history of school violence; 
major staffing changes; leadership 
instability; school restructuring; 
school boundary changes; prior 
district-wide reform efforts; district 
boundary changes; school openings 
or closures; school reorganizations; 
demographic shifts; leadership 
instability; school board relationships 

C_Reform 

history 

S 

S 

C_Facilities 

Data that refer to the condition of 

school facilities or changes that have 

been made that relate to school 

facilities (like moving to a new 
facility). 

New building, more lighting, 
upgraded desks/furniture, dirty or 
worn down, new playground 


V 

V 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

C_Funding 

Data that refer to the overall 

amounts of money reaching the 

school or district. Data could relate to 

levels of sufficiency, changes to 
funding levels over time, various 
funding streams and the levels of 
funding associated with each one. 

May be used in conjunction with 
Sustainability codes in Year 3 to 

discuss how SIG activities will be 

continued. Should not be used with 

SIG_Funding unless respondent is 
discussing how SIG funds fit within 
the larger funding context in the 
school, district, or state. 

Lack of funding, funding from other 
grants or state funding, decline in 
funding, district funding, budget cuts 

C_Level of 
funding 

V 

V 

C_Location 

Data that describe school or district 

geography; defining characteristics of 

a school or district that are related to 

its geography/location. Includes 

whether the school is situated in a 

setting that would be categorized as 
rural and also any transportation 

issues that are related to the school 

or district's location. 

Urban, isolated, surrounded by public 
housing, district very spread out, rural 

V 

V 

V 

C_Misc 

Data that reference school context 

issues that are not adequately 
covered through other codes, but 

describe an element of school context 

that may be important to understand. 



No equivalent 

code 

No equivalent 

code 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

C_Policy 

Environment 

Data that describe state, district, or 
local policy that impact activities at 

the school site or activities related to 

SIG and SIG implementation. Also 
includes data on policies that impact 
ELL instruction and supports (e.g., 
policies that govern how ELLs are 
identified or exited from ELL status, 
what types of instructional models 

are used for ELLs and how those 

models must operate, how ELLs are 
placed into classes or instructional 
programs, what 

standards/assessments are used). 

District politics, school board or 
community-level activities, state or 
district requirements to use particular 
types of ELL programming (e.g., 
mainstreaming, co-teaching) 

C_Local politics 

*State 

* District 

*School 

V 

V 

C_Race 

relations 

Data that reference situations or 

conditions that exist at a school or 

within a district because of issues 

related to race. 

Gangs (if race-related), tensions 
between parents or students of 
different races, school violence, 

tension between teachers and 

students 

V 

No equivalent 

code 

No equivalent 

code 

C_Size 

Data that describe how size of 

classes, schools, or districts are 
important to school context. 

Large class sizes, small school, large 
school, declining enrollment 

S 



C_Stability 

Data that capture degree of stability 

at the school- or district-level across a 

variety of constructs including 
funding, leadership, teacher 
population, school board, reform 
policies or change strategies. Student 
mobility/stability issues are captured 
under student demographics. 

Continuity of funding, leadership, 
students, teachers 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

C_Student 

population 

Data on the types of students that 

attend the school or district and their 

breakdown by subgroup and poverty- 
levels. Also, the degree to which 
student mobility is an issue. 

Includes race, socioeconomic status, 
ELL, special education, socio- 
emotional challenges faced by 
students, family composition 

C_Student 

demographics 

V 

V 
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School Climate ("): Codes in this category apply to segments of data that reference the climate within the school. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

"Attendance 

Data that refer to attendance of students 

and staff in the school. Could be 

challenges like high student or teacher 
absenteeism or the opposite. 

High teacher absenteeism, many 
teachers on sick or maternity leave, 
low student attendance rates, 
frequent need for substitute 

teachers 

C_Attendance 

Y 

Y 

"Buy-in 

Data that discuss the extent to which 

teachers are "bought in" or in support of 
school or district policies, reform agendas, 
leadership approaches, administrators, 

other teachers 

Lack of support for new principal, 
highly supportive of new literacy 

curriculum 

%Buy-in 

Y 

Y 

A Collaboratio 

n 

Data that reference professional learning 
opportunities that focus on building 

collaboration between school 

stakeholders. This could be formal or 

more informal. It may be between only 
teachers, but could also be between 

teachers and administrators. Structured 

opportunities for common planning time 
for teachers either within or across grade 
level or subject. 

Department team meetings, school 
leadership teams, common 
planning time 

$Collaboration 

Y 

Y 

"Community 

Involvement 

Data capturing the extent to which 
community members/groups are involved 

in school activities. Can be from various 

stakeholders' perspectives (teachers, 
principals, parents, students). Data that 
discuss strategies for improved 
involvement or strategies that have 
already improved involvement should also 

be included. 

Strategies for involving community 
members, community members 
attend town hall meetings, 
community members support the 
school, history of community 
involvement, community liaisons 

$Engagement 

*Community 

Y 

Y 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

"Expectations 

for staff 

Data on district, school, community, 
student expectations for staff. Changes to 
these expectations. 

Principals have high expectations 
for teachers, teachers are expected 
to improve student outcomes, 
teachers are expected to work after 
school/on weekends, students look 
for emotional support from 

teachers 

No equivalent 

code 

Y 

Y 

"Expectations 

for students 

Data on district, school, community, 
family expectations for students. Changes 
to these expectations. 

Teachers have high expectations, 
teachers lower expectations to 
meet students' performance level 

No equivalent 

code 

Y 

Y 

A Parent 

Involvement 

Data capturing the extent to which 
parents at the school are involved in 
school community and/or their students' 

education. Can be from various 

stakeholders' perspectives (teachers, 
principals, parents, students). Data that 
discuss strategies for improved 
involvement or strategies that have 
already improved involvement should also 

be included. 

Strategies for involving parents, 
parents attend teacher 
conferences, parents are 
responsive to calls/emails from 
teachers, parents attend classes 
offered by school, parent liaisons 

$Engagement 

*Parent 

Y 

Y 

"School 

Safety 

Data that refer to the safety/orderliness 
of the school environment only within the 
school. Also includes whether safety 
issues have changed over time. 

Violent incidence rates, security 
guards, environment specialists, 
security devices, metal detectors 

C_Safety 

Y 

Y 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

A Staff 

Engagement 

Data capturing the extent to which staff at 
the school are engaged or not in their 

work. Can be from various stakeholders' 

perspectives (teachers, principals, 
parents, students). Change in levels of 
engagement over time or because of 

certain activities. 

Staying long hours, dedication and 
investment to work, willingness to 
do unpaid work 

No equivalent 

code 

No equivalent 

code 

Y 

"Student 

Behavior 

Descriptions of current or past trends in 

behavior of students at the school. Can be 

general school-wide behavior or also data 
related to student behavior by subgroup. 

Includes student behavior issues that are 

the result of racial tensions or evidence of 

positive student behaviors. 

School bullying, fights between 
students based on race, issues with 
suspension or expulsion of 
students, classroom management 

C_Behavior 

Y 

Y 

"Student 

Engagement 

Data capturing the extent to which 
students at the school are engaged both 
academically and extra-curricularly. Can 

be from various stakeholders' 

perspectives (teachers, principals, 
parents, students). Data that discuss 
strategies for improved engagement or 
strategies that have already improved 
engagement should also be included. 

Strategies for improved student 
engagement or evidence of existing 
levels of student engagement. 
Includes: students are prepared for 
class and involved in the lessons, 
students attend tutoring sessions, 
students attend Saturday school or 
summer school, involvement in 
sports teams or other activities 

$Engagement 

*Student 

Y 

Y 

"Student 

Supports 

Activities or resources offered to students 

to improve their overall academic 
performance or to address socio- 
emotional challenges. 

School counselor, help hotlines, 
tutoring programs, after-school 
enrichment programs, social 
workers, medical services 

$Student 

Supports 

Y 

Y 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

A Teacher 

Morale 

Data capturing the entire spectrum of 
teacher morale at a school— high to low. 
Related to buy-in in that data describing 
instances of low morale among staff. 

Improved teacher morale, decline 
in morale, school pride 

*Morale 

*Teacher 

S 

V 

A Trust 

Data that describe the mutual respect 
among teachers, school leaders, staff, 
parents, and community. Data can 
describe trust processes, how it is being 
established, or if it needs to be 

established to move forward with school 

improvement efforts. 

Principal trust in teachers; parent 

trust in school 

No equivalent 

code 

No equivalent 

code 
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Sustainability: Codes in this category apply to segments of data related to the sustainability of improvements and of specific activities. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

Sustainjmpr 

ovements 

Perceptions about the likelihood that 
school, district, or specific 

stakeholders will be able to sustain 

outcomes related to school 

improvement. If no improvement in 
school outcomes were reported over 
the course of SIG but respondents 
talked about activities that they were 
engaged in that they perceived 
would likely lead to positive 
outcomes if sustained passed SIG, 

include those comments here. Also 

include all discussions about 

rationale for coded perceptions. 

Student outcomes, culture/climate 
improvements, improved instruction, 
improved leadership 

No equivalent 

code 

SCSustainability 

V 

Sustain_Steps 
being taken 

How district or school activities being 
paid for by SIG funding will be (or will 
not be) continued in the future. Any 
data that touches on the plans being 
taken to keep SIG-related activities 
or specific processes that schools and 
districts will use to adapt SIG 
activities post-SIG. 

Looking to district for funding; 
writing grants for funding; 
reorganizing school schedule 

No equivalent 

code 

SC_Sustainability 

V 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

Sustain_Activi 

ties 

Data on the specific activities or 
practices that SIG schools are 
choosing to sustain or not, and why 

these choices have been made. 

Related to Sustainjmprovements so 
be mindful about the overlap and 
double-code when necessary. 

Includes specific staff hired through 

SIG. 

Extended day, instructional coaches, 
work with external partner, 
technology adoption 

No equivalent 

code 

SCSu sta i n a b i 1 ity 
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TEACHER-SPECIFIC (T): These codes apply to segments of data that are associated with individual teacher participants. Primarily, they are used 
to capture teachers' goals for their students as well as data on how teachers are gauging success in their classrooms. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

T_Goals for 

students 

Data that reference 

teachers' descriptions of 
the goals they've 

established for their 

students. 

Standardized test 

performance, literacy, 
high school graduation, 
socio-emotional goals 


No equivalent code 

No equivalent code 

T_Measuring 

success 

Data that reference how 

teachers measure their 

students' success in 

meeting the goals that 
they established for 

them 

Benchmark assessments, 
formative assessments, 

informal tools 

V 

No equivalent code 

No equivalent code 

T_Teacher 

quality 

Data that reference 

students, teachers, and 
parents' perceptions 

about what makes a 

high-quality teacher. This 
question comes up 
mostly in the student 

and teacher focus 

groups. 

Group work, patience, 
attention to all students, 
ability to explain 
concepts in multiple 
ways, encouraging, 
supportive, 

communicates well with 

parents 

V 

No equivalent code 

No equivalent code 
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MULTIPURPOSE CODES (*): Codes in this category apply to segments of data from individual interviews and focus groups. These codes are used 
primarily in conjunction with codes from other categories to specify specific stakeholder groups involved in activities or constructs being 
discussed (e.g., school leader/district leader/teacher) or capture constructs that fall outside of the remaining categories. NOTE: There is no 
School multipurpose code because the school is the unit of analysis and assumed to be the default by study participants. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

♦Change 

Strategy 

Data that reference over- 
arching school or district-wide 
change strategies meant to 
"turnaround" low-performing 
schools, or more targeted 
change actions and strategies 
focused on a specific area of 
reform (e.g., curriculum or data 
use). 

Curriculum, data use, student 
supports, change school culture, 
improve instruction 


School change 
codes (See SC 
section above) 

School change 
codes (See SC 
section above) 

♦District 

Leader 

Data that reference high-level 
district leaders, such as 
superintendent or director of 
elementary education. Used in 

addition to other codes on the 

list for segments of data that 
are specific to the district leader 
(personal background of district 
staff), or reflect a perspective 
on district leadership (either by 
teachers, principal, parents, 
students, or other school staff). 

Role definition, capacity, 
problem definition 

V 

SSDistrict 

code 

SSDistrict 

code 

♦Other 

Funding 

Data that reference activities 

taking place at the school or 
district that are not funded by 

SIG. 

Title 1; state improvement 
grants; grants for ELL or Special 

Education 

V 

No equivalent 

code 

No equivalent 

code 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

*Other School 

Staff 

Data that reference other 

school staff (e.g., social worker, 
counselor) roles and 
responsibilities, capacity to 
support schools, and 
perceptions of effectiveness or 
usefulness of these staff by 

other school stakeholders. 

Efficacy of school counselors, 

role of school social workers or 

safety attendants 

V 

No equivalent 

code 

No equivalent 

code 

*Perceived 

Expertise 

Data that reference the extent 

to which specific school or 

district staff have the 

background, training, and 
knowledge that are necessary 
and appropriate for them to do 
their jobs successfully. Can be a 
respondent's own perceptions 
of their expertise or someone 
else's perceptions of them. 

These perceptions must be 
explicitly discussed by study 
participants, not inferred by 
analysts based on data. 

Past leadership experience of 
principal, extent to which 
literacy and math coaches have 
received training to coach other 

teachers in these content areas 


No equivalent 

code 

No equivalent 

code 

*Planned 

Activities 

Data that reference school, 

district or state activities that 

are planned but will be 
implemented in future years. 

Evaluation systems, professional 
learning, new curriculum 


No equivalent 

code 

No equivalent 

code 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

^Rationale 

Data that reference 

stakeholders' rationales for 

making choices about SIG 
and/or any other decisions 
being made at the school or 

district levels. This includes 

respondents' descriptions of 
their own rationales, but also 
respondents' descriptions of 

others' rationales. 

Rationale about staffing, 
curricula, SIG applications, SIG 
model selection, data use, etc. 

V 

No equivalent 

code 

No equivalent 

code 

*Role 

Definition 

Data that reference how study 
participants perceive/describe 
their job role and/or 
responsibilities. Can also include 
descriptions of how other 
stakeholders perceive the roles 
of their colleagues/peers. For 
example, this code could be 
used for a description of how an 
instructional coach perceives 
their job role, but also a 
description of how a principal 
perceives the job role of the 

same coach. 

Job role, duties, responsibilities, 
differences in expectation of 
role versus reality of job 

V 

No equivalent 

code 

No equivalent 

code 

*Strength 

Data that references something 
that is considered a strength for 
either themselves personally or 

the school or district as 

organizations. 



No equivalent 

code 

No equivalent 

code 
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Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

*Student 

Data that reference 

perspectives on students in a 

district or school. 

Morale, buy-in, academic 
performance, commitment to 
school, engagement 

V 

No equivalent 

code 

No equivalent 

code 

*Teacher 

Data that reference 

perspectives on teachers in a 

district or school in addition to 

teachers' roles and 

responsibilities. 

Role definition, teacher 
background, teacher capacity, 
teacher morale, teacher buy-in, 
staffing, data use, collaboration 

V 

No equivalent 

code 

No equivalent 

code 

*Unclear 

Funding 

Data that reference activities 

taking place at the school or 
district where a specific funding 

source cannot be identified. 

Only use if the respondent is 
explicit about the fact that they 
don't know where the funding 
for a specific activity is coming 

from 


No equivalent 

code 

No equivalent 

code 
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SCHOOL BACKGROUND (!): Codes in this category apply to the interviews or focus groups. Attach these codes to the top of the document once. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

lElementary 

Interview or focus group code 
that refers to the type or level 

of school. 

Schools that serve grades K-8 or 
any derivation of these grades 


Incorporated 

into data file 

name 

Incorporated 

into data file 

name 

! High School 

Interview or focus group code 
that refers to the type or level 

of school. 

Schools that serve grades 9-12 
or any derivation of these 
grades 


Incorporated 

into data file 

name 

Incorporated 

into data file 

name 

! Restart 

Interview or focus group code 

that refers to the SIG restart 

model. 

As categorized in SIG subgrant 
application 

S 

No equivalent 

code 

No equivalent 

code 

IRural 

Interview or focus group code 

that refers to schools and 

districts in rural area. 

As categorized in 2008-09 

Common Core of Data 

S 

No equivalent 

code 

No equivalent 

code 

ITransformati 

on 

Interview or focus group code 

that refers to the SIG 

transformation model. 

As categorized by SIG subgrant 
application 

S 

No equivalent 

code 

No equivalent 

code 

ITurnaround 

Interview or focus group code 

that refers to the SIG 

turnaround model. 

As categorized by SIG subgrant 
application 

V 

No equivalent 

code 

No equivalent 

code 

lUrban 

Interview or focus group code 

that refers to schools in urban 

areas (including urban fringe). 

As categorized in 2008-09 

Common Core of Data 

V 

No equivalent 

code 

No equivalent 

code 
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DIMENSION CODES (%)*: Codes in this category apply to segments of data that address key first-year dimensions that are more analytic and less 
descriptive. These codes can be applied to data from all stakeholder groups at the state, district, and school levels. These codes are used to "tag" 
data related to divergence. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

%Divergence 

Data that reference the extent to which 

activities are similar or different from 

activities that the school has implemented 
in the past. Determining similarity can 
involve issues of intensity or newness. 

Continuation of an existing 

reform in a more intense 

manner, implementation of an 
entirely new reform 

Y 

No equivalent 

code 

No equivalent 

code 


* Two other dimension codes used in year 1— buy-in and coherence— were included under different categories in subsequent years. Please reference the School Change 
category and the School Climate category. 


TAGS (&): These codes are used to identify specific questions and answers from the protocol that will contribute to more complex secondary 
analysis. 


Code 

Definition 

Examples 

Year 1 

Year 2 

Year 3 

&Lessons 

Learned 

Data corresponding to questions asked 

about lessons learned. 

Spring 2013: District Protocol Question 16 

Fall 2012: Principal Protocol Questions 5-6, 
Coach Protocol Question 11, and District 

Protocol Questions 9-10 

No 

equivalent 

code 

No 

equivalent 

code 

Y 

&Story Line 

Data corresponding to questions like, "If 
you were to tell the improvement story of 
this school/these schools over the past 
three years, what would the story line be?" 

Spring 2013: External Provider Question 8, 
Principal Protocol Question 19, District 

Protocol Question 10 

No 

equivalent 

code 

No 

equivalent 

code 

Y 

&Theory of 

Action 

Data corresponding to questions specifically 
related to a school's theory of action. 

A separate list of these questions was 
provided to coders 

No 

equivalent 

code 

No 

equivalent 

code 

Y 
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Appendix B. Technical Approach to Qualitative 
Analyses 


Exhibit B.l. 

Perceived Change in Quality of Principal Leadership 


Summary 

This analysis examines perceptions of the change in quality of principal leadership for the 20 
core sample schools that replaced their principal in Year 1 or Year 2 of SIG. See Chapter 3 for a 
discussion of the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 1 and Year 2 interviews with teachers and instructional coaches, including the following 
questions to elicit responses on changes in the quality of principal leadership (Note that 
information may also have been obtained through other points in the interview, not just in 
direct response to the questions listed below.): 

Teachers 

• How would you describe the principal? What do you think are his/her strengths? In 
what areas could he/she improve? 

Instructional coaches 

• How would you describe the principal? What do you think are his/her strengths? 
His/her weaknesses? 

Year 1 and Year 2 focus groups with school improvement teams, teachers, and parents, 
including the following questions to elicit responses on changes in the quality of principal 
leadership (Note that information may also have been obtained through other points in the 
focus group, not just in direct response to the question listed below. Also, the parent focus 
groups did not include specific questions on principal leadership.): 

School improvement teams 

• Who are the key school leaders? What role do the key leaders play? Have there been 
any changes since last year? Why? 

Teachers 

• Who are the key school leaders? What role do the key leaders play? Have there been 
any changes since last year? Why? 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine perceptions of changes in 
the quality of principal leadership, site visitors responded to the following question(s) in the 
online repository based on all coded data associated with principal leadership [! Principal 
Leadership]: 

• [Year 1] Please describe the conditions under which the principal came to this 
school. For example, did he or she come to the school willingly? Did the district 
impose the principal on the school? How soon before the start of the school year did 
the principal arrive? Please note the respondents who provided data on this topic 
and if there were differing perspectives. 

• [Year 1] How did stakeholders describe the leadership style of the principal? 
Centralized or more distributed? Other features? Please identify the respondents 
who provided data on features of principal leadership. 

• [Year 1] In general, was the principal perceived as a catalyst for change in the 
school? Did stakeholders attribute any signs of initial progress to the principal's 
leadership? Please note the respondents who provided data on perceptions of the 
principal's contribution to the change process. 
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Exhibit B.l. 

Perceived Change in Quality of Principal Leadership (continued from previous page) 



Technical Detail 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

(continued from 
previous page) 

• [Year 2] Please describe the current principal's leadership style. How does the 
principal interact with teachers, office staff, and students? Does the principal spend 
time in classrooms, hallways, and his/her office? How would you describe the 
principal's level of morale (e.g., optimistic, frustrated, apathetic)? To what extent 
does the principal develop leaders? 

• [Year 2] Based on your description of the principal's leadership style, please consider 
all of the options listed below and use the text box to indicate which respondents 
described the principal in this manner. 

o Accessible/welcomes input 
o Supportive of staff 
o Visionary 
o Enthusiastic 
o Communicative 
o Develops leaders 
o High expectations 
o Instructional leaders 
o Other 

• [Year 2] If applicable, please describe the circumstances that led to the change in 
principals. Please include where the previous principal was employed in 2011-12, 
and where the current principal was hired from, if known. If there is not sufficient 
information on this topic, please indicate so below. Make sure to include both fall 

2011 and spring 2012 data in your response. 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. 

Stage 2: 

School 

Classification 

Procedures 

Using the responses to the questions above, analysts categorized schools using the 
classification scheme on perceptions of changes in the quality of principal leadership 
described below. When the classifications were complete, the lead site visitor for each school 
reviewed and verified the categorizations for his or her school(s). In cases of disagreement, 
the analysts and lead site visitor returned to the coded data for the particular school(s) in 
question to resolve the disagreement. For this analysis, respondents refer to individuals from 
all of the respondent groups listed above (excluding newly hired staff). 

Quality of principal leadership improved 

• Respondents in at least two respondent groups described the change in principal in 
positive terms; AND 

• No more than one respondent described the change in principal in negative terms. 
Quality of principal leadership did not change 

• Respondents did not comment on the quality of the change in principal leadership or 
described the change in neutral terms. 

Quality of principal leadership declined 

• Respondents in at least two respondent groups described the change in principal in 
negative terms; AND 

• No more than one respondent described the change in principal in positive terms. 

Caveats 

These school-level classifications do not include objective indicators of principal leadership 
quality. Rather, they are based on an aggregate reflection of respondent perceptions of 
changes in the quality of principal leadership. 

Notes 

Includes 20 core sample schools that had a new principal in Year 1 or Year 2 of SIG (2010-11 
or 2011-12). 
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Exhibit B.2. 

Impetus for Principal Replacement 


Summary 

This analysis examines whether principal replacement was perceived as a deliberate effort to 
improve capacity for the core sample schools that replaced their principal for Year 2 of SIG. 

See Chapter 3 for a discussion of the analysis, including analytic results. 

Technical Detail 

Data Sources 

Year 2 interviews with district administrators , principals, teachers, and instructional coaches, 
including the following questions to elicit responses on principal replacement (Note that 
information may also have been obtained through other points in the interviews, not just in 
direct response to the questions listed below.): 

District administrators 

• Why do you think the principal [at each of the core sample schools in the district] was 
selected for the school? 

Principals 

• What are the circumstances that brought about the change in principal at your 
school this year? 

Teachers 

• Have there been any significant changes in the school since the fall that we should 
know about? Please describe. 

Instructional coaches 

• Have there been any changes in the leadership at the school? 

Year 2 focus groups with teachers, including the following questions to elicit responses on 
principal replacement (Note that information may also have been obtained through other 
points in the focus group, not just in direct response to the question listed below.): 

Teachers 

• Who are the key school leaders? What role do the key leaders play? Have there been 
any changes since last year? Why? 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine perceptions regarding the 
impetus for principal replacement, site visitors responded to the following question in the 
online repository based on all coded data associated with principal leadership and staffing 
[(Principal Leadership, $Staffingj: 

• If applicable, please describe the circumstances that led to the change in principals. 
Please include where the previous principal was employed in 2011-12, and where 
the current principal was hired from, if known. If there is not sufficient information 
on this topic, please indicate so below. 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. 
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Exhibit B.2. 

Impetus for Principal Replacement (continued from previous page) 



Technical Detail 

Stage 2: 

School 

Classification 

Procedures 

Using the responses to the question above, analysts categorized schools using the 
classification scheme on impetus for principal replacement described below. When the 
classifications were complete, the lead site visitor for each school reviewed and verified the 
categorizations for his or her school(s). In cases of disagreement, the analysts and lead site 
visitor returned to the coded data for the particular school(s) in question to resolve the 
disagreement. 

Principal replaced as a deliberate effort to improve capacity 

• A district administrator or at least one school-level respondent (principal, teacher, or 
instructional coach) reported that the principal was replaced to improve the capacity 
of the school (e.g., previous principal "was a poor fit for the job"). 

Principal replaced for other reasons 

• A district administrator or at least one school-level respondent (principal, teacher, or 
instructional coach) reported that the principal was replaced because the previous 
principal left for personal reasons or as a result of a promotion. 

Notes 

Includes 9 core sample schools that had a new principal in Year 2 of SIG (2011-12). 
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Exhibit B.3. 

Perceptions of Distributed Leadership in Core Sample Schools 


Summary 

This analysis examines the extent to which core sample schools were perceived as having the 
structures and opportunities to support distributed leadership. See Chapter 3 for a discussion 
of the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with principals, teachers, instructional coaches, and external support 
providers, including the following questions to elicit responses on leadership and decision- 
making (Note that information may also have been obtained through other points in the 
interview, not just in direct response to the questions listed below.): 

Principal 

• How do you view your role as principal? 

• In reflecting on your leadership role, what do you believe are your strengths? In what 
areas do you think you could improve? 

• Please tell me about the school's leadership team (e.g., assistant principals, coaches, 
teacher leaders, etc.) and how you work with them. What is your role in selecting and 
developing the leadership team? 

• Who else provides leadership in this school? 

• How do decisions get made within the school? As the principal, do you make the final 
decisions about key aspects of the school, such as staffing, scheduling, curriculum, 
professional learning, and budget? Why or why not? 

Teachers and instructional coaches 

• What role does the principal play at the school? 

• How would you describe the principal? What do you think are his/her strengths? 

His/her weaknesses? 

• Who are the other leaders at the school, and what role(s) do he/she/they play? How 
would you characterize the school's leadership? 

• Have there been any changes in the leadership team and the roles of its members 
since the fall? Please describe. 

• To what extent can staff/teachers at this school get involved in school decision making? 
What avenues are available to teachers to provide their input? Please describe. 

• In what ways does the school leadership support you in your work? 

External support providers 

• What role does the principal play at the school? 

• How would you describe the principal? What do you think are his/her strengths? In 
what areas could he/she improve? 

• Who are the other leaders at the school, and what role(s) do he/she/they play? Do you 
have any interaction with them? 

• What role does each member of the leadership team play? Have there been any 
changes since last year? 

Year 2 focus groups with school improvement teams and teachers, including the following 
questions to elicit responses on leadership and decision-making (Note that information may 
also have been obtained through other points in the focus group, not just in direct response to 
the question listed below.): 

School improvement team 

• What is the mission of the school improvement team (SIT)? Has the mission changed 
since the 2011-12 school year? If so, how? Why? 

• What have been the SIT's specific responsibilities this school year? 
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Exhibit B.3. 

Perceptions of Distributed Leadership in Core Sample Schools (continued from previous page) 



Technical Detail 

Data Sources 

(continued 
from previous 
page) 

• To what extent has the work of the SIT influenced the school's policies and 
improvement strategies? Please provide examples. Has this changed from last year? 
How? Why? 

• Who are the key school leaders? What role do the key leaders play? Have there been 
any changes since last year? Why? 

Teachers 

• Who are the key school leaders? What role do the key leaders play? Have there been 
any changes since last year? Why? 

To what extent can staff/teachers at this school get involved in school decision making? What 
avenues are available to teachers to provide their input? 

Stage 1: 
Identifying 

Features of 

Distributed 

Leadership 

Based on the literature (Coburn et al., 2003; Lewis et al., 2010), the study team identified the 
following six features associated with distributed leadership: 

• Clear set of responsibilities for school leadership team 

• School leadership team membership open to nonadministrative school staff 

• School leadership team involvement in decision-making process 

• Nonteaching staff (e.g., assistant principals, instructional coaches) involvement in 
decision-making process 

• Teacher involvement in decision-making process 

• Opportunities for teachers to assume leadership roles 

Stage 2: 
Qualitative 

Data 

Analysis 

Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see Chapter 

2 for a description of the coding procedures). To examine the school's leadership structures 
and decision-making processes, site visitors responded to the following questions in the online 
data repository based on all coded data associated with governance and leadership 
[IGovernance, ! Principal Leadership, ITeacher Leadership, lOther Leadership, SSJnstructional 
Coach, SS_External Provider, and A Collaboration]: 

• Please describe the school's governance structure, including who provides 
instructional and administrative leadership at the school— remember to explain who 
provided evidence for this and why they identified certain people as leaders at the 
school. Include a brief description of how decisions are made (including formal and 
informal structures for decision-making), which stakeholders are involved, and 
perceptions of the effectiveness of the governance structure. 

• Is there a leadership team at the school? 

o Yes 

o No 

• If there is a leadership team at the school, please indicate when it was established. 

o Prior to SIG 

o Year 1 of SIG 

o Year 2 of SIG 

o Year 3 of SIG (Planned) 

o Not enough information to answer this question 

If the leadership team was established in Year 1 of SIG or after, please explain why it 
was created (e.g., improving communication, increasing buy-in, etc.) 
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Exhibit B.3. 

Perceptions of Distributed Leadership in Core Sample Schools (continued from previous page) 


Stage 2: 

Qualitative 

Data 

Analysis 

Procedures 

(continued 

from previous 

page) 


Technical Detail 

• If there is a leadership team at the school, please describe: 

o Who is on the leadership team? (how many stakeholders, which stakeholders, 
how and why those stakeholders got selected to be on the leadership team) 
o The role of the leadership team at the school. Please give at least two 
examples of the types of decisions that get made by the leadership team. 

If there is no leadership team at the school, please describe other leadership 
structures at the school and how decisions get made at the school. 

• Please provide additional detail on other formal structures of governance at the school 
(e.g., departmental committees, grade-level teams, administrative committees). 

Please explain who is involved in these other governance structures and what role 
these other governance structures play in the school. 

• Please describe informal leadership opportunities at the school (e.g., opportunities for 
teachers to lead professional learning opportunities, to provide input on curriculum, or 
to take on other leadership roles). 

• To what extent are each of the stakeholders below involved in the decision-making 
process at their school? 


O <u 

4-> 

g o jn 



Not involved: Respondents did not mention being 
involved in the decision making process at the 
school 


Somewhat involved: Respondents mentioned 
having input and partaking in some decision- 
making processes at the school, such as helping 
with the master schedule or student groupings 

Highly involved: 

Respondents mentioned having a large say in 
major decisions at the school, such as selecting 
curriculum or selecting the instructional focus for 
a grade level 

Final say: Respondents mentioned having the final 
word or decision-making power 


• Taking into account formal and informal leadership, has the leadership structure or 
practice (who is doing what and how) changed since the beginning of SIG? 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. 
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Exhibit B.3. 

Perceptions of Distributed Leadership in Core Sample Schools (continued from previous page) 



Technical Detail 

Stage 3: 

School 

Classification 

Procedures 

For each feature associated with distributed leadership, analysts first determined whether the 
school exhibited the feature using the following decision rules: 

• Clear set of responsibilities for school leadership team. At least one member of the 
school leadership team reported that the team has a clear set of responsibilities. 

• School leadership team membership open to the nonadministrative school staff. At 
least one member of the school leadership team suggested that the school leadership 
team is open to nonadministrative school staff. 

• School leadership team involvement in decision-making process. The principal 
indicated that the school leadership team is involved in decision making. 

• Nonteaching staff involvement in decision-making process. A respondent from at least 
two of the following respondent groups— assistant principal, instructional coach, or 
other school administrator— indicated that the principal makes decisions with input 
from nonteaching school staff. 

• Teacher involvement in decision-making process. At least two teachers indicated that 
the principal makes decisions with input from teachers. 

• Opportunities for teachers to assume leadership roles. At least two teachers indicated 
that there are leadership opportunities for teachers. 

Analysts then classified the core sample schools based on the classification scheme on 
distributed leadership described below. Because there were no natural breaks in the 
distribution, cut points were set to divide the schools roughly into thirds. When the 
classifications were complete, the lead site visitor for each school was required to review and 
verify the categorizations for his or her school(s). In cases of disagreement, the analysts and 
lead site visitor returned to the coded data for the particular school(s) in question to resolve 
the disagreement. 

Structures and opportunities to support distributed leadership 

• Perceived to exhibit at least five of the six features associated with distributed 
leadership. 

Moderate structures and opportunities to support distributed leadership 

• Perceived to exhibit three or four of the six features associated with distributed 
leadership. 

Little/no structures or opportunities to support distributed leadership 

• Perceived to exhibit less than three of the six features associated with distributed 
leadership. 

Analysts also reviewed the data— specifically the questions about changes between Year 1 and 
Year 2 of SIG— to determine whether or not the structures to develop distributed leadership 
existed prior to SIG and if principals undertook efforts to develop distributed leadership during 
Year 1 or Year 2 of SIG. Schools were classified as having made efforts to build infrastructure to 
support distributed leadership during SIG if at least one respondent from the respondent 
groups listed above provided a clear statement regarding the year of a new structure. 

Notes 

Includes 25 core sample schools. Data from Year 2 of SIG was supplemented with data and 
findings from Year 1 of SIG. 
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Exhibit B.4. 

Teaching Staff Stability in Core Sample Schools 


Summary 

This analysis examines the amount of turnover of the teaching staff in the core sample 
schools at the beginning of Years 1 and 2 of SIG implementation. See Chapter 4 for a discussion 
of the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with district administrators , principals, teachers, and instructional coaches, 
including the following questions to elicit responses on the stability of the teaching staff. 

(Note that information may also have been obtained through other points in the interview, 
not just in direct response to the questions listed below.): 

District administrators 

• Have the schools experienced changes related to staffing— principals and teachers— 
since last year? Please describe. What were the reasons for any changes? 

Principal 

• What was the school's experience with teacher turnover at the start of this school 
year? 

Teachers 

• Have there been any significant changes in the school since last year? Please 
describe. 

Instructional coaches 

• To what extent have there been changes at your school from last year to this year? 
Please explain. 

Year 2 focus groups with teachers, including the following questions to elicit responses on the 
stability of the teaching staff (Note that information may also have been obtained through 
other points in the focus group, not just in direct response to the questions listed below.): 
Teachers 

• Have there been any significant changes in the school since last year? Please 
describe. 

Findings from the Year 1 SST report on the analysis of improvement actions implemented in 
core sample schools (see Exhibit B.8 in Le Floch et al., 2014) 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine the stability of the 
teaching staff, site visitors responded to the following question in the online data repository 
based on all coded data associated with staffing [$Staffingj: 

• What was the school's experience with staff changes (and/or stability) in the 2011- 
12 school year (and why)? Please include context from 2010-11 (Year 1 of SIG) and 
earlier, as needed. 

• What proportion of the school's teachers was new in Year 2 of SIG, according to 
respondents? How much teacher turnover did the school experience? (If the school 
had teacher position openings that it could not fill, please note that as well.) 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. 
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Exhibit B.4. 

Teaching Staff Stability in Core Sample Schools (continued from previous page) 



Technical Detail 

Stage 2: 

School 

Classification 

Procedures 

Using the responses to the Year 2 repository questions and findings from the Year 1 report, 
analysts categorized schools based on the classification scheme on teacher stability described 
below. When the classifications were complete, the lead site visitor for each school reviewed 
and verified the categorizations for his or her school(s). In cases of disagreement, the 
analysts and lead site visitor returned to the coded data for the particular school(s) in 
question to resolve the disagreement. For this analysis, respondents refer to individuals from 
all of the respondent groups listed above. 

Greater than half of teachers each year 

• At least two respondents reported that at least 50 percent of the teaching staff was 
new at the beginning of both Years 1 and 2 of SIG. 

Less than half of teachers in one year, and more than half of teachers in the other year 

• At least two respondents reported that at least 50 percent of the teaching staff was 
new at the beginning of Year 1 but not Year 2, or vice versa. 

Less than half of teachers each year 

• At least two respondents reported that less than 50 percent of the teaching staff 
was new at the beginning of both Years 1 and 2 of SIG. 

Caveats 

The data collected in Year 1 did not allow a more detailed analysis of stability than the 50 
percent threshold. In addition, these school-level classifications are not based on a review of 
teacher roster data, but rather on respondent perceptions of changes in the teaching staff. 

Notes 

Includes 25 core sample schools. 
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Exhibit B.5. 

Perceptions of the Teacher Replacement Process 


Summary 

This analysis examines perceptions of the teacher replacement process (i.e., the rules governing 
the hiring and removal of staff and the extent to which principals and teachers were involved in the 
replacement process) for the core sample schools that replaced at least half of their teaching staff 
in Year 2 of SIG. See Chapter 4 for a discussion of the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with district administrators, principals, and teachers, including the following 
questions to elicit responses on the teacher replacement process (Note that information may 
also have been obtained through other points in the interview, not just in direct response to 
the questions listed below.): 

District administrator 

• Is there a district-wide approach to instructional improvement planned/being 
implemented for the SIG schools? What specific strategies are involved in this 
approach? Please explain. 

• For each school, what instructional improvement strategies were 
planned/implemented this school year? What is the rationale behind these 
strategies? 

• In SIG schools in which the principal and many staff members were replaced, how 
were decisions made about which staff to keep and which to let go? What was the 
rationale? 

• Do you have any strategies in place to recruit and retain high quality/effective 
teachers and principals? To improve the knowledge and skills of teachers and 
principals through professional learning? 

Principal 

• Could you describe the specific improvement strategies your school has implemented 
this school year? Flow have you prioritized these strategies? 

• Were any teachers or other staff replaced? If so, when and which staff were 
replaced, and why? 

• Flow would you describe the staff in this school? 

Teachers 

• Are there some broad approaches or strategies that the school as a whole is 
following to reach its improvement goals? What are they, and do you think they are 
appropriate or likely to be effective? 

• Can you describe some of the specific improvement strategies that you know are, or 
will be, adopted at your school this school year as part of SIG? 

• Flow would you describe the teaching staff at this school? What are their strengths 
and weaknesses as a staff? 

Year 2 focus groups with teachers, including the following questions to elicit responses on the 
teacher replacement process (Note that information may also have been obtained through 
other points in the focus group, not just in direct response to the questions listed below.): 

Teachers 

• Can you describe some of the specific improvement strategies that were, or will be, 
adopted at your school this school year? 

• Do you think the improvement strategies fit the needs of the school and/or 
students? 

• What will be the greatest challenges to implementing these strategies? 
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Exhibit B.5. 

Perceptions of the Teacher Replacement Process (continued from previous page) 



Technical Detail 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine respondent perceptions of 
the teacher replacement process, site visitors responded to the following questions based on 
the coded data (specific codes used are provided in brackets): 

• Please categorize your school along the continuum using the check boxes below if at 
least three respondents provide evidence that the school belongs in the specific 
category. [$Staffing, IStaff Capacity] 

o New staff were perceived as skilled/motivated, bringing energy and 

expertise into the school, and building the school's human capacity. At the 
same time, prior staff may be perceived as unskilled/unmotivated, 
o New staff were perceived as unskilled/unmotivated, detrimental to the 
school in a way that limits its instructional capacity, 
o Changes in staff were perceived as maintaining the status quo, not changing 
the human capital capacity of the school. 

• Please add details about your response above, citing evidence and sources. 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy. 

Stage 2: 

School 

Classification 

Procedures 

Using the responses to the questions above, analysts categorized schools using the classification 
scheme on perceptions of the teacher replacement process described below. When the 
classifications were complete, the lead site visitor for each school reviewed and verified the 
categorizations for his or her school(s). In cases of disagreement, the analysts and lead site 
visitor returned to the coded data for the particular school(s) in question to resolve the 
disagreement. For this analysis, respondents refer to individuals from all of the respondent 
groups listed above (excluding newly hired staff). 

Positive 

• At least three respondents indicated that the new teachers introduced as part of the 
replacement process were beneficial (i.e., "bringing new energy," improving staff 
morale, increasing teacher quality); AND 

• No respondent described the teacher replacement in terms such as "biased" or 
"unfair." 

Neutral 

• Respondents did not comment on the quality of the teacher replacement process or 
the quality of new teachers, or described the process in neutral terms, such as 
"another instance of change." 

Negative 

• At least three respondents indicated that the new teachers introduced as part of the 
replacement process were detrimental to the school (i.e., weakening staff morale, 
decreasing teacher quality); OR 

• At least three respondents described the teacher replacement process in terms such 
as "biased" or "unfair." 

Caveats 

This analysis is not an examination of teacher effectiveness, and it is not intended to imply a 
causal connection between teacher replacement and changes in teacher effectiveness. 

Rather, this analysis is an aggregate reflection of the perceptions of respondents regarding the 
teacher replacement process. 

Notes 

Includes 2 schools that replaced at least half of their teaching staff in Year 2 of SIG (findings 
for the 9 core sample schools that replaced at least half of their teaching staff in Year 1 are 
taken from the Year 1 SST report [see Le Floch et al., 2014]). 
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Exhibit B.6. 

Creation of Nonteaching Positions in Core Sample Schools 


Summary 

This analysis examines the creation of nonteaching positions in Years 1 or 2 of SIG in core 
sample schools. See Chapter 4 for a discussion of the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 1 and 2 interviews with district administrators, principals, teachers, and instructional 
coaches, including the following questions to elicit responses on new nonteaching staff 
positions. (Note that information may also have been obtained through other points in the 
interview, not just in direct response to the questions listed below.): 

District administrators 

• Year 1: For each school, what instructional improvement strategies are being 
planned/implemented? 

• Year 2: Flave the schools experienced changes related to staffing— principals and 
teachers— since last year? Please describe. 

Principals 

• Year 1: Could you describe the specific improvement strategies your school has 
implemented this school year? 

• Year 1: Flow would you describe the staff in this school? Were there any changes to 
personnel this school year?) 

• Year 2: Flave there been any significant changes in the school since the fall that we 
should know about? Please describe. 

• Year 2: What have been the school's recent experiences attracting/recruiting 
qualified staff? 

Teachers 

• Year 1: Can you describe some of the specific improvement strategies that are, or 
will be, adopted at your school this school year? 

• Year 2: What are the main ways in which the school is working to accomplish its 
goals? What specific activities are being implemented? Please describe. 

Instructional coaches 

• Year 1: Can you describe some of the specific improvement strategies that are being, 
or will be, adopted this school year? 

• Year 2: In our previous conversation, you had mentioned the school is implementing 
the following key improvement strategies. To what extent have these strategies 
changed during this school year? 

Year 1 and 2 focus groups with school improvement teams and teachers, including the 
following questions to elicit responses on new nonteaching staff positions (Note that 
information may also have been obtained through other points in the focus group, not just in 
direct response to the questions listed below.): 

School improvement teams 

• Year 1: Flow do you think the SIG improvement approaches/strategies identified for 
the school will address the issues facing the school? Please describe how you think 
specific improvement strategies are working, or will work, to improve the school. 

• Year 2: In our previous conversation, you had mentioned the school is implementing 
the following key improvement strategies. To what extent have these strategies 
changed during this school year? 
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Exhibit B.6. 

Creation of Nonteaching Positions in Core Sample Schools (continued from previous page) 



Technical Detail 

Data Sources 

(continued from 
previous page) 

Teachers 

• Year 1: Can you describe some of the specific improvement strategies that are, or 
will be, adopted at your school this school year? 

• Year 2: What are the main ways in which the school is working to accomplish its 
goals? What specific activities are being implemented? Please describe. 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine the creation of 
nonteaching positions, site visitors responded to the following questions in the online data 
repository, separately for Year 1 and Year 2, based on all coded data associated with staffing 
[$Staffing] . 

• Were any new nonteaching positions created? 

o Yes 

o No 

• If so, please list the number of new nonteaching positions, by type. 

o Administrator 

o Instructional coach 
o Community/parent liaison 
o Other 

• Please add details about each new position, citing evidence and sources. 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. 

Stage 2: 
Classification 

Procedures 

Using the repository responses to the questions above, analysts determined whether a new 
nonteaching position had been created using the criteria described below. When the analysis 
was complete, the lead site visitor for each school reviewed and verified the results of his or 
her school(s). In cases of disagreement, the analysts and lead site visitor returned to the 
coded data for the particular school(s) in question to resolve the disagreement. For this 
analysis, respondents refer to individuals from all of the respondent groups listed above. 

Position created 

• At least one respondent reported that a new nonteaching position (e.g., 

administrator, coach, community/parent liaison, or other) was created at the school. 

Notes 

Includes 25 core sample schools. 
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Exhibit B.7. 

Principal's Approach to Staffing Decisions in Core Sample Schools 


Summary 

This analysis examines the extent to which principals at core sample schools demonstrated a 
purposeful approach to staffing decisions in Year 2 of SIG. See Chapter 4 for a discussion of the 
analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with district administrators and principals, including the following questions 
to elicit responses on staffing (Note that information may also have been obtained through 
other points in the interview, not just in direct response to the questions listed below.): 

District administrators 

• What have been the district's experiences in attracting/recruiting qualified teachers 
to work in SIG schools? How, if at all, have these experiences differed between SIG 
and non-SIG schools and across SIG schools? 

• What is the process for filling open teacher positions in the SIG schools? Is this any 
different for non-SIG schools? Have there been any changes in the process between 
the last school year (2010-11) and this school year (2011-12)? 

• Do the SIG schools have discretion in identifying and "letting go" ineffective 
teachers? 

• What strategies, if any, exist to encourage qualified staff to remain at the SIG 
schools? If any, when were these strategies first implemented? How do they differ 
from non-SIG school strategies? 

Principals 

• What have been the school's recent experiences attracting/ recruiting qualified 
staff? Is the school able to attract qualified— i.e., skilled and motivated— teachers? 
Why or why not? What factors contribute to these experiences? 

• What strategies, if any, exist to encourage qualified staff to remain at the school? 

Are these unique to the school? 

Year 2 focus groups with school improvement teams (No specific questions were asked. 
Information for this analysis may have been captured at multiple points in the focus group.) 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine the level of purposefulness 
in the principals' approach to staffing, site visitors responded to the following questions in 
the online data repository based on all coded data associated with staffing [$Staffing] : 

• What was the school's experience with staff changes (and/or stability) in the 

2011-12 school year and why? Be sure to mention the principal, teachers, and 
coaches (or other staff as needed). Please note that this includes turnover, 
intentional staffing changes (hiring/removal), as well as changes in staff 
responsibilities/assignments within the school (includes changes to get staff into the 
"right spot," as well as more disruptive staff responsibility changes). 

• Please describe the school's and/or its district's strategies to recruit and retain 
qualified staff, noting the extent to which the school (and/or union) is able to 
leverage SIG to do so. Please note the extent to which the school expects strategies 
to address (or not address) recruitment and retention needs, and please explain 
why. Also, indicate whether these actions are or are not special to SIG schools in the 
district and/or other similar schools (i.e., low-performing, rural) in the district. Cite 
evidence and sources. 
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Exhibit B.7. 

Principal's Approach to Staffing Decisions in Core Sample Schools 

(continued from previous page) 



Technical Detail 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

(continued from 
previous page) 

• According to respondents, to what extent was any change in the school's staff 
(teachers, principals, coaches, or other staff) part of a purposeful strategy to build 
the school's staff capacity (that is, the collective skills/knowledge of adults at the 
school)? Please describe the change and extent to which decisions were motivated 
(or perceived to be motivated) by school needs (e.g., need for a more skilled/ 
motivated staff, need for staff in a certain subject area, etc.), and who was making 
those decisions. 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. 

Stage 2: 

School 

Classification 

Procedures 

Using the principal reports identified in the repository responses and coded data from the 
principal interviews, analysts categorized schools based on the classification scheme on the 
purposefulness of the principal's staffing approach, described below. Data from other 
respondents, including district administrators and school improvement teams, provided 
details about the principal's approach to staffing decisions. When the classifications were 
complete, the lead site visitor for each school reviewed and verified the categorizations for 
his or her school(s). In cases of disagreement, the analysts and lead site visitor returned to 
the coded data for the particular school(s) in question to resolve the disagreement. 

Purposeful approach to staffing decisions 

• The principal clearly articulated how staffing decisions addressed school goals or 
needs (e.g., removing a teacher due to poor performance and replacing her with 
someone with the "right" skill set and strong motivation, or reassigning teachers to 
different grades to better align their skills to students 1 needs; creating an additional 
instructional coach position to give teachers "the support that they need"). 

No evidence of purposeful approach to staffing decisions 

• The principal clearly articulated how staffing decisions were not aligned with school 
goals or needs (e.g., accepting low-quality teachers due to district mandate); OR 

• The principal did not provide any evidence on how staffing decisions addressed 
school goals or needs (e.g., replacing teachers leaving for personal reasons such as 
retirement or relocation). 

Caveats 

This analysis uses principal interviews as the primary source. Thus, this analysis is not 
intended to be an objective measure of the schools' approach to staffing decisions. In 
addition, this analysis is limited to staffing decisions that occurred during Year 2 of SIG, and 
thus excludes intentional staffing decisions made in Year 1, which may have continued to 
address school goals or needs in Year 2. 

Notes 

Includes 25 core sample schools. 
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Exhibit B.8. 

Perceived Teacher Retention and Recruitment Challenges in Core Sample Schools 


Summary 

This analysis examines perceptions of teacher recruitment and retention challenges in the 
core sample schools. See Chapter 4 for a discussion of the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with principals and district administrators, including the following questions 
to elicit responses on staff recruitment and retention challenges (Note that information may 
also have been obtained through other points in the interview, not just in direct response to 
the questions listed below.): 

Principal 

• What have been the school's recent experiences attracting/recruiting qualified 
staff? Has this changed since last year? If so, how? Why or why not? How, if at all, is 
the process for filling open teacher positions in the school different from non-SIG 
schools? Please explain. 

• What was the school's experience with teacher turnover at the start of this school 
year? Do you anticipate there will be any teacher turnover and/or instability of staff 
next year? 

District administrator 

• What have been the district's experiences in attracting/recruiting qualified teachers 
to work in SIG schools? How, if at all, have these experiences differed between SIG 
and non-SIG schools and across SIG schools? 

• Do the SIG schools have discretion in identifying and "letting go" ineffective 
teachers? Please describe the process. 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine respondent perceptions of 
staff recruitment and retention challenges, site visitors responded to the following questions 
in the online data repository based on all coded data for the principal and district 
administrator interviews associated with staffing [$Staffing] and staff capacity [IStaff 
capacity]: 

• Please describe the extent to which staff recruitment and/or retention is a challenge 
(includes principal, teachers, coaches, other staff as needed). Why is it a challenge? 
Please be sure to differentiate between recruitment and retention and indicate the 
source(s) of evidence. 

• Please check any of the below that apply to the school's experience with 
recruiting/retaining teachers, as indicated by the district or principal. 

o Lengthy commutes for teachers 

o Poor reputation of school 

o Stressful school environment due to poor student behavior 
o Stressful school environment due to administration/staff 
o Stressful school environment due to safety issues 
o Open teacher positions at the start of the school year 
o School compelled to hire teachers previously removed from another school 
in the district or that were left over in the applicant pool (i.e., among the last) 
o Teaching staff included Teach for America (TFA) teachers 
o Other 

o None of the above 

Please provide details on the challenges, citing evidence and sources. 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. 
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Exhibit B.8. 

Perceived Teacher Retention and Recruitment Challenges in Core Sample Schools 

(continued from previous page) 



Technical Detail 

Stage 2: 
Classification 

Procedures 

Using the principal and district administrator reports in the repository responses, for each 
challenge, analysts identified whether the explanation was perceived as a teacher 
recruitment or retention challenge using the criteria described below. Data from other 
school-level respondents provided details about the types of staff recruitment and retention 
challenges faced. When the analysis was complete, the lead site visitor for each school 
reviewed and verified the results of his or her school(s). In cases of disagreement, the 
analysts and lead site visitor returned to the coded data for the particular school(s) in 
question to resolve the disagreement. 

Identified as a perceived staff recruitment or retention challenge 

• The principal and/or district administrator(s) described this explanation as a staff 
recruitment or retention challenge. 

Caveats 

This analysis is not a systematic examination of staffing recruitment and retention 
challenges, but rather a reflection of the perceptions of district administrators and principals 
regarding recruitment and retention challenges at their schools. 

Notes 

Includes 25 core sample schools. 
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Exhibit B.9. 

Types of Support for Staff Recruitment and Retention in Core Sample Schools 


Summary 

This analysis examines the types of supports for improving staff recruitment and retention in 
core sample schools in Year 2 of SIG. See Chapter 4 for a discussion of the analysis, including 
analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with district administrators and principals, including the following questions 
to elicit responses on supports for staff recruitment and retention (Note that information 
may also have been obtained through other points in the interview, not just in direct 
response to the questions listed below.): 

District administrators 

• Are any incentives provided to attract teachers to the SIG schools? If so, are these 
incentives also offered for other schools in the district? 

• What is the process for filling open teacher positions in the SIG schools? Is this any 
different for non-SIG schools? Have there been any changes in the process between 
the last school year (2010-11) and this school year (2011-12)? Please explain. 

• What strategies, if any, exist to encourage qualified staff— principals and teachers— 
to remain at the SIG schools? If any, when were these strategies first implemented? 
How do they differ from non-SIG school strategies? 

Principals 

• What strategies, if any, exist to encourage qualified staff to remain at the school? 

Are these unique to the school? 

• Are any incentives provided to attract teachers to the school? If so, please describe. 

• How, if at all, is the process for filling open teacher positions in the school different 
from non-SIG schools? Please explain. 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine the types of supports for 
improving staff recruitment and retention, site visitors responded to the following questions 
in the online data repository based on all coded data for the principal and district 
administrator interviews associated with staffing [$Staffing] and staff capacity [ IStaff 
capacity]: 

• For all schools, whether or not teacher retention is a challenge, what 

actions/strategies have been implemented to improve/promote retention, as 
indicated by the district or principal? 

o Implement specific actions with the goal to improve teacher satisfaction 
with job (e.g., honoring teachers, allowing flexibility, improving teacher- 
only space) 

o Offer incentives for staying 

o Offer advancement/leadership opportunities for teachers 
o Offer supports for teacher commute 
o Other 

o None of the above 

Please provide details on the actions/strategies, citing evidence and sources. 
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Technical Detail 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

(continued from 
previous page) 

• For all schools, whether or not teacher recruitment is a challenge, what 

actions/strategies have been implemented for attracting quality teachers to teach at 
the school, as indicated by the district or principal? 

o Implement specific actions with the goal to improve school reputation 
(e.g., staff attending community events to discuss school) 
o Offer incentives for coming 

o Provide SIG schools priority in hiring and selecting teachers 
o Offer supports for teacher commute 

o Widen recruitment efforts to outside of district, to other communities, 
teaching schools, etc. 
o Other 

o None of the above 

Please provide details on the actions/strategies, citing evidence and sources. 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. Using the responses to the questions above, analysts identified schools that 
had particular support to retain and/or recruit their teachers. Coded data from other 
respondents, such as teachers and coaches, were used for illustrative purposes. 

Stage 2: 
Classification 

Procedures 

Using the principal and district administrator reports identified in the repository responses, 
for each type of support, analysts identified whether the support was being provided using 
the criteria described below. When the analysis was complete, the lead site visitor for each 
school reviewed and verified the categorizations for his or her school(s). In cases of 
disagreement, the analysts and lead site visitor returned to the coded data for the particular 
school(s) in question to resolve the disagreement. 

Identified as a support 

• The principal and/or district administrator(s) reported that this support was used to 
retain and/or recruit teachers. 

Caveats 

This analysis focuses on whether specific structures and supports to improve teacher 
recruitment and retention were offered in core sample schools. This analysis is not an 
examination of the effectiveness of these supports. 

Notes 

Includes 25 core sample schools. 


B-20 


Case Studies of Schools Receiving School Improvement Grants: Final Report 


Exhibit B.10. 

Perceptions of the Alignment Between Professional Learning Opportunities and 
School Goals and Needs in Core Sample Schools 


Summary 

This analysis examines the extent to which professional learning opportunities were 
perceived as being aligned with school goals and needs. See Chapter 5 for a discussion of 
the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with district administrators, principals, teachers, instructional coaches, and 
external support providers, including the following questions to elicit responses on the 
alignment between professional learning opportunities and school goals and needs (Note 
that information may also have been obtained through other points in the interview, not 
just in direct response to the questions listed below. External support providers were not 
explicitly asked questions specific to professional learning.): 

District administrators 

• Are there district-level strategies to increase the capacity for leaders in SIG 
schools? 

• Thinking about last year and contrasting it with this year, what changes, if any, can 
be noted with regard to professional learning opportunities for school leaders in 

SIG schools? 

• Are there district-level strategies to increase the capacity for teachers in SIG 
schools? 

• Thinking about last year and contrasting it with this year, what changes, if any, can 
be noted with regard to professional learning opportunities for teachers in SIG 
schools? 

Principals 

• What are the primary professional learning opportunities available to teachers? 

How do these activities fit into the overall strategy for improvement? 

• Have the professional learning opportunities changed since the last school year? 

• In your role as principal, have you had any professional learning opportunities this 
school year? Has this changed since the last school year? 

Teachers 

• What are the primary professional learning opportunities in which you and other 
teachers take part? 

• To what extent have you been able to use what you learned in the professional 
learning opportunities in your classroom? What has been the most valuable to 
you? Why? What has not worked well for you and why? 

Instructional coaches 

• Do you have a role in determining the need for professional learning opportunities 
and/or providing these opportunities to teachers? 

• Thinking of this school year, are the professional learning opportunities available at 
this school different from what's offered at other schools? 

• This school year, what has been particularly strong about the professional learning 
opportunities offered? 
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Technical Detail 

Data Sources 

(continued from 
previous page) 

Focus groups with school improvement teams and teachers, including the following 
questions, asked during Year 2 of SIG, to elicit responses on the alignment between 
professional learning opportunities and school goals and needs (Note that information may 
also have been obtained through other points in the focus group, not just in direct response 
to the questions listed below. School improvement teams were not explicitly asked 
questions specific to professional learning.): 

Teachers 

• What are the primary professional learning opportunities in which you took part 
this school year? 

• To what extent have you been able to use what you learned in the professional 
learning opportunities in your classroom? What has been the most valuable to 
you? Why? What has not worked well for you and why? 

Stage 1: 

Qualitative Data 

Analysis 

Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 
Chapter 2 for a description of the coding procedures). To examine the extent to which 
professional learning opportunities were perceived as being aligned with school goals and 
needs, site visitors responded to the following questions in the online data repository based 
on all coded data associated with professional learning [$Professional Learning]: 

• As described by teachers, the principal, and the instructional coach(es), what were 
the primary professional learning opportunities in which teachers took part during 
the 2011-12 school year? Please note which respondent(s) provided evidence and 
detail any instances of disagreement between respondents. 

• For the primary professional learning opportunities described above, please 
describe (a) who was involved with determining the need for the activity/activities, 
(b) how areas for professional learning were identified, and (c) the rationale for the 
activity/activities (i.e., compliance with federal, state, or district mandates; 
addressing identified performance problems; targeted to performance of teachers 
or students; focused on interests of individual teachers, etc.). Please note which 
respondent(s) provided evidence and detail any instances of disagreement 
between respondents. (For this question, respondents refer to the principal, school 
improvement team, teachers, instructional coach(es), external support provider(s), 
and district administrator(s).) 

• Considering the evidence provided by respondents on the rationale for 
professional learning opportunities, please rate your school based on the extent to 
which overall professional learning opportunities are "aligned"— that is, the extent 
to which professional learning opportunities are tied to systemwide and/or school- 
specific performance goals, plans, and needs. 

o Aligned. Schools categorized as having "purposeful" professional learning 
opportunities are those in which most or all professional learning activities 
are motivated by outcomes for teachers or students, such as addressing 
the performance or learning needs of staff members, the performance 
goals for students, or the improvement plans of the school or district, 
o Moderately aligned. Schools categorized as having "moderately 

purposeful" professional learning opportunities are those in which some, 
but not most, professional learning opportunities are motivated by the 
outcomes for teachers or students, such as addressing the performance or 
learning needs of staff members, the performance goals for students, or 
the improvement plans of the school or district. 
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Technical Detail 

Stage 1: 

Qualitative Data 

Analysis 

Procedures 

(continued from 
previous page) 

o Not aligned. Schools categorized as having "minimally purposeful" or "not 
purposeful" professional learning opportunities are those in which the set 
of professional learning activities do not address the learning needs of the 
staff members as a whole nor do they tie to the performance goals or 
improvement plans of the school or district. 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. 

Stage 2: 

School 

Classification 

Procedures 

Using the repository responses to the questions above, analysts categorized schools using 
the classification scheme on the alignment between professional learning opportunities and 
school goals and needs described below. When the classifications were complete, the lead 
site visitor for each school reviewed and verified the categorizations for his or her school(s). 

In cases of disagreement, the analysts and lead site visitor returned to the coded data for 
the particular school(s) in question to resolve the disagreement. For this analysis, 
respondents refer to individuals from all of the respondent groups listed above. 

Aligned 

• At least one respondent in an administrative or support role (e.g., district 
administrator, principal, instructional coach, external support provider) and at least 
two teachers described professional learning opportunities, as a whole, as focused 
on the goals or needs of the school; AND 

• Not more than one respondent explicitly described the professional learning 
opportunities, as a whole, as not focused on the goals or needs of the school. 

Moderately aligned 

• At least one respondent (but fewer than one administrator and two teachers) 
described the professional learning opportunities as focused on the goals and 
needs of the school, AND no respondent explicitly described the professional 
learning opportunities as not focused on the goals or needs of the school; OR 

• At least two respondents described professional learning as a mix of opportunities, 
some that were focused on the goals or needs of the school and some that were 
not; OR 

• Respondents disagreed about the extent to which professional learning 
opportunities, as a whole, are focused on the goals or needs of the school. 

Minimally or not aligned 

• Not more than one respondent described the professional learning opportunities, 
on the whole, as focused on the goals or needs of the school, AND at least two 
respondents explicitly described the professional learning as not focused on the 
needs of the school. 

• Not more than one respondent described the professional learning opportunities, 
on the whole, as focused on the goals or needs of the school, AND no respondent 
articulated any connections between professional learning opportunities and the 
goals or needs of the school. 

Caveats 

This analysis is not an objective examination of the alignment between professional learning 
opportunities and school goals and needs, but rather is an aggregate reflection of the 
perceptions of respondents regarding professional learning. 

Notes 

Includes 25 core sample schools. 


B-23 


Case Studies of Schools Receiving School Improvement Grants: Final Report 
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School-Level Structures to Support Teacher Collaboration 


Summary 

This analysis examines whether core sample schools established structures to support 
teacher collaboration. See Chapter 5 for a discussion of the analysis, including analytic 
results. 


Technical Detail 

Data Sources 

Year 2 interviews with principals, teachers, instructional coaches, and external support 
providers, including the following questions to elicit responses on teacher collaboration 
(Note that information may also have been obtained through other points in the interview, 
not just in direct response to the questions listed below. Principals and external support 
providers were not explicitly asked questions specific to teacher collaboration.): 

Teachers 

• At the school, to what extent do teachers have the opportunity to collaborate and 
work together? Is this different from last year? How? 

Instructional coaches 

• To what extent do teachers have the opportunity to collaborate and work together? 

Is this different from last year? How? 

Year 2 focus groups with teachers, including the following questions to elicit responses on 
teacher collaboration (Note that information may also have been obtained through other 
points in the focus group, not just in direct response to the questions listed below.): 

Teachers 

• At the school, to what extent do teachers have the opportunity to collaborate and 
work together? Is this different from last year? How? 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine structures to support 
teacher collaboration, site visitors responded to the following questions in the online data 
repository based on all coded data associated with collaboration [ A Collaboration]: 

• Please describe the structures related to common planning and collaboration, 
including a summary of the structures, the respondents who described them, and 
whether they were funded by SIG. 

• What was the rationale for this approach to planning or collaboration? Please note 
the rationale and the respondents. 

• Please describe whether respondents attributed improved capacity or constrained 
capacity to the planning or collaboration structures. 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. 
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Technical Detail 

Stage 2: 
Classification 

Procedures 

Using the repository responses to the questions above, analysts identified whether 
structures were in place to support teacher collaboration using the criteria described below. 
When the analysis was complete, the lead site visitor for each school reviewed and verified 
the categorizations for his or her school(s). In cases of disagreement, the analysts and lead 
site visitor returned to the coded data for the particular school(s) in question to resolve the 
disagreement. For this analysis, respondents refer to individuals from all of the respondent 
groups listed above. 

Identified as having structures to support teacher collaboration 

• At least one teacher and respondents in at least one other respondent group 
specifically described one or more of the following: scheduled weekly or monthly 
time for common planning or study groups, guidelines or protocols for efficient use 
of collaborative time, or other opportunities for peer-to-peer discussion of 
instruction or content; AND 

• Respondents in at least two respondent groups specifically noted that structures to 
support teacher collaboration were new in Year 1 (2010-11) or Year 2 (2011-12) of 
SIG, or the SIG budget provided evidence that SIG supported this structure; AND 

• No respondent explicitly reported that structures to support teacher collaboration 
had been eliminated or reduced in Years 1 or 2 of SIG. 

Notes 

Includes 25 core sample schools. 
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School-Level Structures to Support Data Use 


Summary 

This analysis examines whether core sample schools established structures to support data 
use. See Chapter 5 for a discussion of the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with principals, teachers, instructional coaches, and external support 
providers, including the following questions to elicit responses on data use (Note that 
information may also have been obtained through other points in the interview, not just in 
direct response to the questions listed below.): 

Principals 

• To what extent have the improvement strategies you are implementing changed 
during this school year? 

• How are teachers using student-level data to modify instructional practices? What 
are the data sources (e.g., student work embedded in the classroom, standardized 
tests, benchmark assessments, grades, attendance, other)? Why? 

Teachers 

• Do you use student-level data in your classroom(s)? How has your use of data 
changed during the past year, and why? 

• What are the main ways in which the school is working to accomplish its goals? 

What specific activities are being implemented? 

Instructional coaches 

• Did your role and responsibilities change over the course of the year? If yes, how 
and why? 

• Do you use student-level data in your work with teachers? Why or why not? 

• To what extent have the improvement strategies changed during this school year? 

External support providers 

• Do you use school-level and/or student-level data in your work with this school? 

Why or why not? If so, please explain. 

Year 2 focus groups with teachers, including the following questions to elicit responses on 
data use (Note that information may also have been obtained through other points in the 
focus group, not just in direct response to the questions listed below.): 

Teachers 

• What are the main ways in which the school is working to accomplish its goals? 

What specific activities are being implemented? 

• Over the course of this school year, did you receive support from others? Did you 
work with a coach? With the external support provider? 
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Technical Detail 

Stage 1: 

Qualitative Data 

Analysis 

Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 
Chapter 2 for a description of the coding procedures). To examine structures to support 
data use, site visitors responded to the following questions in the online data repository 
based on all coded data associated with collaboration [$Data Use]: 

• Please describe the structures related to data use, including a summary of the 
structures, the respondents who described them, and whether they were funded 
bySIG. 

• What was the rationale for these data collection or management structures? 

Please note the rationale and the respondents. 

• Please describe whether respondents attributed improved capacity or constrained 
capacity to the data use structures. 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. 

Stage 2: 
Classification 

Procedures 

Using the repository responses to the questions above, analysts identified whether 
structures were in place to support data use using the criteria described below. When the 
analysis was complete, the lead site visitor for each school reviewed and verified the 
categorizations for his or her school(s). In cases of disagreement, the analysts and lead site 
visitor returned to the coded data for the particular school(s) in question to resolve the 
disagreement. For this analysis, respondents refer to individuals from all of the respondent 
groups listed above. 

Identified as having structures to support data use 

• Respondents in at least two respondent groups specifically described one or more 
of the following: access to diagnostic or benchmark assessments, an online data 
management system and early warning system, or external supports for data 
analysis; AND 

• Respondents in at least two respondent groups specifically noted that structures to 
support data use were new in Year 1 (2010-11) or Year 2 (2011-12) of SIG, or the 

SIG budget provided evidence that SIG supported this structure; AND 

• No respondent explicitly reported that structures to support data use had been 
eliminated or reduced in Years 1 or 2 of SIG. 

Notes 

Includes 25 core sample schools. 
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Exhibit B.13. 

District Support Services to Build Human Capital in Core Sample Schools 


Summary 

This analysis examines the types of support services to improve human capital received from 
districts in core sample schools. See Chapter 6 for a discussion of the analysis, including 
analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with district administrators, principals, and teachers, including the following 
questions to elicit responses on the district's role in building school-level human capital 
(Note that information may also have been obtained through other points in the interview, 
not just in direct response to the questions listed below.): 

District administrators 

• How did the district support the SIG schools this year? How was this different from 
the last school year? Why? 

• Are there particular ways in which the district has built its own capacity to support 

SIG implementation this year? 

• Are there any district-level strategies to increase the capacity for leaders in SIG 
schools? 

• Are there any district-level strategies to increase the capacity for teachers in SIG 
schools? 

Principals 

• During this school year, did the district provide you with guidance or input into the 
appropriate school improvement strategies? 

• What is the process for filling open teacher positions or removing ineffective 
teachers from the school? Is that different from the process in other district 
schools? What is the district role? 

Teachers 

• What are the primary professional learning opportunities in which you took part this 
school year? 

• Have the professional learning opportunities available to you this school year 
changed from what was available last year? If so, how? 

• Over the course of this school year, did you receive support from others? Did you 
work with the coach? With the external provider? Please describe. 

• Do you have sufficient support to integrate the use of data in your classroom? Why 
or why not? If yes, please describe how you are supported. 

Year 2 focus groups with teachers, conducted during Year 2 of SIG, including the following 
questions to elicit responses on the district's role in building school-level human capital 
(Note that information may also have been obtained through other points in the focus group, 
not just in direct response to the questions listed below.): 

Teachers 

• What are the primary professional learning opportunities in which you took part this 
school year? 

• Have the professional learning opportunities available to you this school year 
changed from what was available last year? If so, how? 

• Over the course of this school year, did you receive support from others? Did you 
work with the coach? With the external provider? Please describe. 

• Do you have sufficient support to integrate the use of data in your classroom? Why 
or why not? If yes, please describe how you are supported. 


B-28 


Case Studies of Schools Receiving School Improvement Grants: Final Report 


Exhibit B.13. 

District Support Services to Build Human Capital in Core Sample Schools 
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Technical Detail 

Stage 1: 

Identifying 

District 

Supports 

Based on a review of the literature on the district role in building school capacity, the study 
team identified the following support services, which were categorized into four broad 
categories or domains— interpreting and using data; teacher staffing policies (recruitment 
and retention); teacher professional learning; and building school leadership capacity: 

• Interpreting and using data 

o Benchmark assessments 
o Supplying data to school site 

o Providing data analysis tools 

• Teacher staffing policies (recruitment and retention) 

o Supportive teacher staffing policies 

• Teacher professional learning 

o District-led teacher professional learning 
o District-supported instructional coaches 

• Building school leadership capacity 

o Providing leadership coaching or other district-level professional learning 
support for school principals 

o Establishing systems that allow principals to problem solve and collaborate 
towards shared objectives 

Stage 2: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine the types of support 
services that core sample schools received from their district, site visitors responded to the 
following question in the online data repository based on all coded data associated with 
district support and the district policy environment [SS District, C_Policy Environment]: 

• Please describe the district's general approach to supporting capacity-building in 
low-performing schools. If possible, please explain the district's rationale or theory 
of action underlying this approach. 

In addition, for each category of support services, site visitors were asked to identify 
respondent groups that reported receiving a particular support from the district, as well as to 
provide additional details on the support, including the nature and focus of the support; 
duration and frequency with which the support was provided; the scope of the support (e.g., 
SIG schools only, targeted to schools on some other basis, offered districtwide); rationale for 
providing the support; the extent to which the support aligned with district- and school-level 
priorities and goals; source of funding (e.g., SIG, other source); and the extent to which 
contextual factors (e.g., teacher's union, community relations, other district policies) 
facilitated or detracted from the district's ability to build capacity through the support. 
Questions for supports related to interpreting and using data are provided below as an 
example of the repository questions on district support services to build human capital: 

• For each support service associated with interpreting and using data, please identify 
whether the following respondents identified this support as being offered by the 
district (check as many as apply). [SS District, $Data Use, $Evaluation, 

SC_Monitoring Change, SC_Alignment of Practices] 
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Technical Detail 

Stage 2: 
Qualitative 

Data Analysis 
Procedures 

(continued from 
previous page) 

ij 1 ji I 

Benchmark Assessments 

Supplying data to school site 

Prowling data analysis toots 

Other (please specify) 

Note: For respondent groups with ’nuftble respondents, art least one respondent must ha-#e reported a 

particular support as oeing provided 

• Please use the space below to describe the supports that are being provided. Include 
the following types of information in your description and please document your data 
sources: brief description of support (nature and focus of the support, frequency with 
which the support is provided, rationale for providing the support); source of funding 
(SIG/Other); duration of support (pre-SIG, SIG); whether support is only for SIG 
schools, or whether it is offered district wide, or targeted to schools on some other 
basis (e.g., to schools with a particular accountability status); whether support aligns 
with perceptions of district priorities/goals; whether support aligns with perceptions 
of school-level priorities/goals; whether there are contextual factors (union role, 
community relations, other district policies) that facilitate or detract from the district's 
ability to successfully build capacity using this activity. [SS District, $Data Use, 
$Evaluation, SC_Monitoring Change, SC_Alignment of Practices] 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. 

Stage 3: 
Classification 

Procedures 

Using the repository responses to the questions above, for each category of district support, 
analysts identified whether the support was being provided using the criteria described 
below. When the classifications were complete, the lead site visitor for each school reviewed 
and verified the categorizations for his or her school(s). In cases of disagreement, the 
analysts and lead site visitor returned to the coded data for the particular school(s) in 
question to resolve the disagreement. For this analysis, respondents refers to individuals 
from the following respondent groups: teachers, instructional coaches, other school leaders 
(e.g., school leadership team members, assistant principals), and parents. 

Identified as a type of support 

• The principal and/or a district administrator and a respondent from at least one 
additional respondent group identified one or more specific support service as being 
supplied by the district. For district supports related to teacher staffing and building 
school leadership capacity, the threshold was lowered to reports from the principal 
and/or a district administrator only, as these respondents were in the best position 
to describe these types of supports. 

Caveats 

These analyses cannot determine the district's rationale behind offering certain types of 
services to SIG schools over others. Here, we focus on categorizing the types of services that 
district and school stakeholders reported either providing or receiving. 

Notes 

Includes 22 of 25 core sample schools. Three schools implementing the restart model were 
excluded from this analysis because they are managed by either EMOs or CMOs that have been 
designated as external partners (as opposed to districts) for the purposes of our analyses. 
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District Organizational Structures to Build Human Capital in Core Sample Schools 


Summary 

This analysis examines whether districts established organizational structures to support 
building human capital at core sample schools. See Chapter 6 for a discussion of the analysis, 
including analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with district administrators, principals, instructional coaches, and teachers, 
including the following questions to elicit responses on the district's role in building school- 
level human capital (Note that information may also have been obtained through other 
points in the interview, not just in direct response to the questions listed below.): 

District administrators 

• How did the district support the SIG schools this year? How was this different from 
the last school year? Why? 

• Are there particular ways in which the district has built its own capacity to support 

SIG implementation this year? 

• Are there any district-level strategies to increase the capacity for leaders in SIG 
schools? 

• Are there any district-level strategies to increase the capacity for teachers in SIG 
schools? 

Principals 

• During this school year, did the district provide you with guidance or input into the 
appropriate school improvement strategies? 

• What is the process for filling open teacher positions or removing ineffective 
teachers from the school? Is that different from the process in other district 
schools? What is the district role? 

Instructional coaches 

• Thinking of this school year, is the professional learning available at this school 
different from what's offered at other schools? How? Why? 

Teachers 

• What are the primary professional learning opportunities in which you took part this 
school year? 

• Do you have a sense that the professional learning opportunities available to you at 
this school this year are different from what's offered at non-SIG schools? What is 
different? Why? 

Year 2 focus groups with school improvement teams and teachers, including the following 
questions to elicit responses on the district's role in building school-level human capital (Note 
that information may also have been obtained through other points in the focus group, not 
just in direct response to the questions listed below.): 

School improvement teams 

• What type of assistance has the school received in its efforts to improve the school? 
Has this changed since last year? How? Why? 

Teachers 

• What are the primary professional learning opportunities in which you took part this 
school year? 

• Do you have a sense that the professional learning opportunities available to you at 
this school this year are different from what's offered at non-SIG schools? What is 
different? Why? 
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(continued from previous page) 



Technical Detail 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine district-level organizational 
structures to support school capacity-building, site visitors responded to the following 
questions in the online data repository based on the coded data (specific codes used are 
provided in brackets): 

• Please describe the district's general approach to supporting capacity-building in 
low-performing schools. If possible, please explain the district's rationale or theory 
of action underlying this approach. [SS District, C_Policy Environment] 

• What, if any, structures does the district have in place to support SIG schools 
specifically as opposed to its other schools? [SS District, SIG_Processes, SIG_Funded 
Activities] 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. Using the responses to the questions above, analysts identified the following 
two district organizational structures to build human capital: 

• Specific sub-districts to support low-performing schools, such as SIG schools, in the 
improvement process 

• Specific staff positions (i. e., central office positions or teams) to support low- 
performing schools, such as SIG schools, in the improvement process. 

Stage 2: 
Classification 

Procedures 

Using the repository responses to the questions above, for each district organizational 
structure, analysts identified whether the structure was in place using the criteria described 
below. When the classifications were complete, the lead site visitor for each school reviewed 
and verified the categorizations for his or her school(s). In cases of disagreement, the 
analysts and lead site visitor returned to the coded data for the particular school(s) in 
question to resolve the disagreement. For this analysis, respondents refer to individuals from 
all of the respondent groups listed above. 

Identified as a district organizational structure 

• A district administrator and a respondent from at least one additional respondent 
group identified the structure as being in place. 

Caveats 

While districts may have used other organizational structures to monitor or support schools, 
these analyses focus specifically on sub-district and SIG administrator structures because 
these were a) those identified by study participants and b) referenced in the SIG Guidance. 

Notes 

Includes 13 districts in the core sample. 
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Perceived Usefulness of District Supports in Core Sample Schools 


Summary 

This analysis examines the extent to which respondents perceived the supports (i.e., services, 
policies, and structures) as useful in building human capital and facilitating overall school 
improvement. See Chapter 6 for a discussion of the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with principals, teachers, and instructional coaches, including the following 
questions to elicit responses on the district's role in building school-level human capital (Note 
that information may also have been obtained through other points in the interview, not just 
in direct response to the questions listed below.): 

Principals 

• During this school year, did the district provide you with guidance or input into the 
appropriate school improvement strategies? 

• How does the district measure and monitor your school's success? 

• What are the primary professional learning opportunities available to teachers? How 
do these activities fit into the overall strategy for improvement? 

• In your role as principal, have you had any professional learning opportunities this 
school year? Has this changed since the last school year? How? Why or why not? 

• How are teachers using student-level data to modify instructional practices? What 
are the data sources (e.g., student work embedded in the classroom, standardized 
tests, benchmark assessments, grades, attendance, other?) 

• What is the process for filling open teacher positions or removing ineffective 
teachers from the school? Is that different from the process in other district 
schools? What is the district role? 

Teachers 

• Do you have sufficient support to integrate the use of data in your classroom? Why 
or why not? If yes, please describe how you are supported. 

• What are the primary professional learning opportunities in which you and other 
teachers take part? 

• To what extent have you been able to use what you learned in the learning activities 
in your classroom? What has been the most valuable to you? Why? What has not 
worked well for you and why? 

Instructional coaches 

• Did you receive any specific training or acquire a particular certification to be an 
instructional coach? 

• Do you use student-level data in your work with teachers? Why or why not? If so, 
please explain. 

• Thinking of this school year, are the professional learning opportunities available at 
this school different from what's offered at other schools? 

Year 2 focus groups with school leadership teams and teachers, including the following 
questions to elicit responses on the district's role in building school-level human capital (Note 
that information may also have been obtained through other points in the focus group, not 
just in direct response to the questions listed below.): 

School leadership teams 

• What type of assistance has the school received in its efforts to improve the school? 
Has this changed since last year? How? Why? 
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Technical Detail 

Data Sources 

(continued from 
previous page) 

Teachers 

• What are the primary professional learning opportunities in which you took part this 
school year? 

• To what extent have you been able to use what you learned in the professional 
learning opportunities in your classroom? What has been the most valuable to you? 
Why? What has not worked well for you and why? 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine respondent perceptions of 
how useful their district supports were, site visitors responded to the following questions in 
the online data repository based on the coded data (specific codes used are provided in 
brackets): 

• Please describe the district's general approach to supporting capacity-building in 
low-performing schools. If possible, please explain the district's rationale or theory 
of action underlying this approach. [SS District, C_Policy Environment] 

• Please describe the relationship between your SIG school and its district. Do the 
district respondents feel that the district offers strong support to your SIG school? 
Why/why not? Do school-level respondents agree? Has this relationship changed 
over time/since SIG, or has it been stable? If there have been changes, to what do 
respondents attribute those changes? If there is disagreement about the strength of 
the relationship, why is this the case? Is the district doing anything 
(actions/strategies/policies) that is interfering with or constraining your school's 
ability to build capacity? [SS District, C_District_School History, C_Policy 

Environment, SC_Perceived Change, SIG_Processes, SC_Monitoring Change, 
SC_Challenge] 

• What, if any, structures does the district have in place to support SIG schools 
specifically as opposed to its other schools? [SS District, SIG_Processes, SIG_Funded 
Activities] 

• Please use the space below to describe the supports that are being provided. 

Include the following types of information in your description and please document 
your data sources: brief description of support (nature and focus of the support, 
frequency with which the support is provided, rationale for providing the support); 
source of funding (SIG/Other); duration of support (pre-SIG, SIG); whether support is 
only for SIG schools, or whether it is offered district wide, or targeted to schools on 
some other basis (e.g., to schools with a particular accountability status); whether 
support aligns with perceptions of district priorities/goals; whether support aligns 
with perceptions of school-level priorities/goals; whether there are contextual 
factors (union role, community relations, other district policies) that facilitate or 
detract from the district's ability to successfully build capacity using this activity. 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. 
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Technical Detail 

Stage 2: 

School 

Classification 

Procedures 

Using the repository responses to the questions above, analysts categorized schools using the 
classification scheme on perceptions of district support described below. When the 
classifications were complete, the lead site visitor for each school reviewed and verified the 
categorizations for his or her school(s). In cases of disagreement, the analysts and lead site 
visitor returned to the coded data for the particular school(s) in question to resolve the 
disagreement. Although the data used in this analysis primarily focused on the perceived 
usefulness of district support specifically related to building human capital, in some cases, 
respondents spoke about the usefulness of the district's support as a whole, which may have 
included other types of support. For this analysis, respondents refer to individuals from all of 
the respondent groups listed above. 

Perceived district supports as useful 

• The principal and at least one other respondent indicated that district supports were 
useful to school improvement efforts (For district supports related to teacher 
staffing or building leadership capacity, the threshold was lowered to reports from 
the principal only, as the principal was in the best position to describe this type of 
support.); AND 

• The principal described district administrators as accessible and helpful when 
assisting the school to solve problems and overcome challenges to improvement; 

AND 

• No more than one respondent described district supports as constraining school 
improvement. 

Mixed perceptions of district supports 

• The principal disagreed with at least two other respondents about the usefulness of 
district supports; OR 

• The principal and at least one other respondent indicated that certain district 
supports were useful, while others were not useful. 

Perceived district supports as not useful 

• The principal and at least one other respondent indicated that district supports were 
not established, not useful, or were slowing or preventing school improvement 
efforts; AND 

• The principal described district administrators as inaccessible or difficult to work 
with when assisting the school to solve problems and overcome challenges to 
improvement; AND 

• No more than one respondent described district supports as useful to school 
improvement. 

Caveats 

This analysis is not an objective examination of the effectiveness of the support supplied by 
districts in facilitating school improvement, but rather an aggregate reflection of 
respondents' perceptions of the support's usefulness. Because our data limit us from 
determining whether specific district supports (e.g., instructional coaches) were useful in 
building human capital, this analysis focuses on respondents' overall perceptions of how 
useful their district supports were. 

Notes 

Includes 22 of 25 core sample schools. Three schools implementing the restart model were 
excluded from this analysis because they are managed by either EMOs or CMOs that have been 
designated as external partners (as opposed to districts) for the purposes of our analyses. 
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Areas of External Support Provider Support to Build Human Capital 


Summary 

This analysis examines the types of support services to improve human capital received 
from external support providers in core sample schools. See Chapter 6 for a discussion 
of the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with district administrators, principals, teachers, and instructional 
coaches, including the following questions to elicit responses on the external support 
providers' role in building school-level human capital (Note that information may also 
have been obtained through other points in the interview, not just in direct response 
to the questions listed below.): 

District administrators 

• Are there particular ways in which the district has built its own capacity to 
support SIG implementation this year? Please describe. Is this different from 
last year? How? Why? 

o Has the district worked with an external partner? If so, describe the 
partner's role and if it is different from the prior year. 

Principals 

• What are the primary professional learning opportunities available to 
teachers? How do these activities fit into the overall strategy for 
improvement? 

• In your role as principal, have you had any professional learning opportunities 
this school year? Has this changed since the last school year? How? Why or 
why not? 

• How are teachers using student-level data to modify instructional practices? 
What are the data sources (e.g., student work embedded in the classroom, 
standardized tests, benchmark assessments, grades, attendance, other?) 

Teachers 

• What are the primary professional learning opportunities in which you and 
other teachers take part? 

• Have the professional learning opportunities available to you this school year 
changed from what was available last year? If so, how? 

Instructional coaches 

• Thinking of this school year, are the professional learning opportunities 
available at this school different from what's offered at other schools? How? 
Why? 

Year 2 focus groups with school improvement teams and teachers, including the 
following questions to elicit responses on the external support providers' role in 
building school-level human capital (Note that information may also have been 
obtained through other points in the focus group, not just in direct response to the 
questions listed below.): 

School improvement teams 

• What type of assistance has the school received in its efforts to improve the 
school? Has this changed since last year? How? Why? 

Teachers 

• What are the primary professional learning opportunities in which you took 
part this school year? 
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Technical Detail 

Stage 1: Qualitative 
Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 
Chapter 2 for a description of the coding procedures). To examine the types of 
supports that core sample schools received from external support providers, analysts 
reviewed all coded data from district administrators, principals, teachers, school 
improvement team members, and instructional coaches associated with external 
providers [SS_External Provider]. They then responded to the following questions in 
the online data repository based on the relevant coded data: 

• What expertise is the external provider supposed to bring to the school? 

• How is the external support intended to build capacity of the school? 

• If the school works with an external partner, please describe the role of the 
provider, the focus of the support, and the relationship with stakeholders. 

Using these data, analysts identified the following areas of support considered to build 
the capacity of school leaders and staff: 

• Budget planning, including professional learning for the principal and/or 
school improvement team on how to spend SIG funds. 

• Coaching, including professional learning for instructional coaches. 

• Community involvement, including professional learning for the community 
liaison. 

• Curriculum and instruction, such as professional learning on particular 
content areas (e.g., reading, mathematics). Common Core, and English 
language learners. 

• Data use, such as professional learning on the use of data in planning 
instruction. 

• School culture and climate, such as professional learning for teachers on 
strategies to improve student behavior. 

• School leadership, including professional learning for the principal on topics 
such as SIG and curriculum and instruction; other teacher/staff leadership 
development; and developing governance structures. 

• Staff collaboration, including professional learning on designing school-level 
collaboration structures. 

• Strategic planning, including professional learning on developing the school 
vision, goals, and theory of action, and on identifying school challenges. 

• Staffing, such as assistance in teacher evaluation and in the teacher hiring and 
terminating processes. 

Activities such as evaluation support (if no professional learning element); support on 
vendor tools; assistance with school accreditation; assistance in developing financial 
incentives; monitoring; supports for tutoring, afterschool, and summer school 
programs; and assistance with teacher union negotiations were not considered to be 
human capital-building efforts and thus were excluded from the analysis. 


B-37 


Case Studies of Schools Receiving School Improvement Grants: Final Report 


Exhibit B.16. 

Areas of External Support Provider Support to Build Human Capital 

(continued from previous page) 



Technical Detail 

Stage 2: 

Classification 

Procedures 

Using the coded data, for each type of external provider support, analysts identified 
whether the support was being provided using the criteria described below. When the 
classifications were complete, the lead site visitor for each school reviewed and 
verified the categorizations for his or her school(s). In cases of disagreement, the 
analysts and lead site visitor returned to the coded data for the particular school(s) in 
question to resolve the disagreement. For this analysis, respondents refer to 
individuals from all of the respondent groups listed above. 

Identified as an area of support 

• The principal or a school improvement team member, and a respondent from 
at least one additional respondent group, identified the support as being 
supplied by the external support provider. 

Caveats 

These analyses may not capture all external providers who were present in schools 
with the purpose of building human capital, if respondents failed to mention them or if 
there was insufficient information on the substantive focus. 

Notes 

Includes 25 core sample schools. 
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intensity of SIG Partner Supports 


Summary 

This analysis examines the intensity of supports provided by SIG partners (e.g., external 
support providers that school leaders considered central to the change process under 

SIG) among the 13 core sample schools that identified a SIG partner. See Chapter 6 for a 
discussion of the analysis, including analytic results. 

Technical Detail 

Data Sources 

Year 2 interviews with principals and external support providers, including the following 
questions to elicit responses on supports provided by SIG partners (Note that 
information may also have been obtained through other points in the interview, not just 
in direct response to the questions listed below. Principals were not explicitly asked 
questions specific to SIG partners.): 

External support providers 

• What are your main responsibilities/roles within this school? To whom are you 
responsible? Has your role changed during the last year? 

• Please describe a typical week of work [or visit] to the school this year? Who do 
you work with, and what are your specific activities? Why? What do you hope 
to accomplish? 

Stage 1: 

Qualitative Data 
Analysis 

Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 
Chapter 2 for a description of the coding procedures). To examine the intensity of 
supports provided by SIG partners, analysts reviewed all coded data for the principal and 
external support provider interviews associated with the external support provider 
[SS_External Provider]. 

Stage 2: 

School 

Classification 

Procedures 

Using the coded data, analysts categorized schools using the classification scheme on 
intensity of SIG partner supports described below. When the classifications were 
complete, the lead site visitor for each school reviewed and verified the categorizations 
for his or her school(s). In cases of disagreement, the analysts and lead site visitor 
returned to the coded data for the particular school(s) in question to resolve the 
disagreement. For this analysis, "external support provider" refers to the external 
support provider who served as the school's SIG partner. 

Concentrated 

• The principal or external support provider reported that the SIG partner 
provided supports through one-time professional learning activities. 

Ongoing 

• The principal or external support provider reported that the SIG partner 
provided supports regularly throughout the year. 

Ongoing with additional supports 

• The principal or external support provider reported that the SIG partner 
provided supports that are ongoing with additional supports (i.e., one-time 
professional learning activities; work in summer; remote supports). 

Notes 

Includes 17 external support providers in 13 core sample schools that reported having a 
SIG partner in Year 2 of SIG. 
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Perceived Fit Between SIG Partners and Core Sample Schools 


Summary 

This analysis examines perceptions of the fit between SIG partners (e.g., external 
support providers that school leaders considered central to the change process under 

SIG) and core sample schools among the 13 core sample schools that identified a SIG 
partner. See Chapter 6 for a discussion of the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with principals and external support providers, including the following 
questions to elicit responses on the fit between the SIG partner and the school (Note 
that information may also have been obtained through other points in the interview, not 
just in direct response to the questions listed below. Principals were not explicitly asked 
questions specific to SIG partners.): 

External support providers 

• What is your expertise? What kind of training have you had? 

• What is the background or expertise of the organization? What kind of training 
does your organization provide? 

• How many schools are you personally currently working with? How many are 
low performing? 

• How and when did you come to be the support provider for this school? 

Year 2 focus groups with school improvement teams, including the following questions to 
elicit responses on the fit between the SIG partner and the school (Note that information 
may also have been obtained through other points in the interview, not just in direct 
response to the questions listed below.): 

School improvement teams 

• What type of assistance has the school received in its efforts to improve the 
school? Has this changed since last year? How? Why? 

o Have you felt that the provider understood your school and its needs? 

Stage 1: 

Qualitative Data 
Analysis 

Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 
Chapter 2 for a description of the coding procedures). To examine respondent 
perceptions of the fit between SIG partners and core sample schools, analysts reviewed 
all coded data for the principal and external support provider interviews and school 
improvement team focus groups associated with the external support provider 
[SS_External Provider], 
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Technical Detail 

Stage 2: 

School 

Classification 

Procedures 

Using the coded data, analysts categorized schools using the classification scheme on 
perceived fit of SIG partners described below. When the classifications were complete, 
the lead site visitor for each school reviewed and verified the categorizations for his or 
her school(s). In cases of disagreement, the analysts and lead site visitor returned to the 
coded data for the particular school(s) in question to resolve the disagreement. For this 
analysis, "external support provider" refers to the external support provider who served 
as the school's SIG partner. 

Relevant 

• The external support provider and either the principal or a member of the 
school improvement team described the external support provider as having 
relevant experience, defined as expertise working with the particular school or 
schools with similar characteristics, experience working with teachers or 
administrators, and prior teaching/administrative experience; AND 

• No respondent mentioned any specific shortcomings of the external support 
provider's experience and expertise. 

Moderate 

• The external support provider and either the principal or a member of the 
school improvement team described the external support provider as having 
some relevant experience, but mentioned shortcomings in one of the three 
areas: expertise working with the particular school or schools with similar 
characteristics, experience working with teachers or administrators, and prior 
teaching/administrative experience. 

Low 

• The external support provider and either the principal or a member of the 
school improvement team described the external support provider as not 
having relevant experience, mentioning shortcomings in at least two of the 
three areas: expertise working with the particular school or schools with similar 
characteristics, experience working with teachers or administrators, and prior 
teaching/administrative experience. 

Notes 

Includes 17 external support providers in 13 core sample schools that reported having a 

SIG partner in Year 2 of SIG. 
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Perceived Usefulness of SIG Partner Support 


Summary 

This analysis examines the extent to which respondents perceived the supports provided 
by SIG partners as useful. See Chapter 6 for a discussion of the analysis, including 
analytic results. 


Technical Detail 

Data Sources 

Year 2 interviews with district administrators, principals, teachers, instructional coaches, 
and external support providers, including the following questions to elicit responses on 
the role of the SIG partner (Note that information may also have been obtained through 
other points in the interview, not just in direct response to the questions listed below. 
Principals, teachers, and instructional coaches were not explicitly asked questions 
specific to SIG partners.): 

District administrators 

• How did the district support SIG schools this year? How was this different from 
the last year? What is the district's role in identifying external provider(s) for 
the school(s) and then working with the external provider(s) on school issues? 

• Are there particular ways in which the district has built its own capacity to 
support SIG implementation this year? (If the district has an external partner) 
Describe the partner's role and if it is different from last year. 

External support providers 

• What are your main responsibilities/roles within this school? To whom are you 
responsible? Has your role changed during the last year? 

• What is your role in the implementation of the key improvement strategies 
being implemented by the school this school year? 

Year 2 focus groups with school improvement teams and teachers, including the 
following questions to elicit responses on the role of the SIG partner (Note that 
information may also have been obtained through other points in the focus group, not 
just in direct response to the questions listed below. Teachers were not explicitly asked 
questions specific to SIG partners.): 

School improvement teams 

• What type of assistance has the school received in its efforts to improve the 
school? Has this changed since last year? How? Why? 

o Have you felt that the provider understood your school and its needs? 

Stage 1: 

Qualitative Data 
Analysis 

Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 
Chapter 2 for a description of the coding procedures). To examine respondent 
perceptions of SIG partner support's usefulness, analysts reviewed all coded data 
associated with the external support provider [SS_External Provider]. 
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Technical Detail 

Stage 2: 

Classification 

Procedures 

Using the coded data, analysts categorized schools using the classification scheme on 
perceptions of SIG partner supports described below. When the classifications were 
complete, the lead site visitor for each school reviewed and verified the categorizations 
for his or her school(s). In cases of disagreement, the analysts and lead site visitor 
returned to the coded data for the particular school(s) in question to resolve the 
disagreement. Although the data used in this analysis primarily focused on the 
usefulness of SIG partner support specifically related to building human capital, in some 
cases, respondents spoke about the usefulness of the SIG partner's support as a whole, 
which may have included other types of support. For this analysis, respondents refer to 
individuals from all of the respondent groups listed above. 

Perceived SIG partner supports as useful 

• At least two respondents described the SIG partner in predominately positive 
terms; AND 

• No more than one respondent holds a negative perception of the SIG partner. 

Mixed perceptions of SIG partner supports 

• Respondents disagreed with one another about the usefulness of supports 
provided by the SIG partner; OR 

• Respondents indicated that certain supports provided by the SIG partner were 
useful, while others were not useful. 

Perceived SIG partner supports as not useful 

• At least two respondents described the SIG partner in predominantly negative 
terms; AND 

• No more than one respondent holds a positive perception of the SIG partner. 

Notes 

Includes 15 of 17 external support providers in 13 core sample schools that reported 
having a SIG partner in Year 2 of SIG. Two external support providers were excluded 
from this analysis due to insufficient data. 
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Overall Efforts to Build Human Capital in Core Sample Schools 


Summary 

This analysis examines the overall level of efforts to build human capital among core 
sample schools in Year 2 of SIG. See Chapter 7 for a discussion of the analysis, including 
analytic results. 

Technical Detail 

Data Sources 

Year 1 interviews with district administrators, principals, teachers, and instructional 
coaches 

Year 1 focus groups with school improvement teams and teachers 

Year 2 interviews with district administrators, principals, teachers, instructional coaches, 
and external support providers 

Year 2 focus groups with school improvement teams and teachers 

Year 2 teacher survey data 

Stage 1: Identifying 
Indicators 

From the analyses included in Chapters 3 through 6, the study team identified the 
following ten indicators to gauge the overall level of efforts to build human capital for 
each core sample school (Technical approach exhibits associated with the indicators are 
provided in parentheses. Mean hours of teacher-reported professional learning are 
based solely on teacher survey data and thus do not have a technical approach exhibit.): 

• Efforts to build structures for distributed leadership (Exhibit B.3) 

• Replacement of 50 percent of teachers (Exhibit B.4) 

• Creation of nonteaching staff positions (Exhibit B.6) 

• Purposeful approach to staffing (Exhibit B.7) 

• Alignment of professional learning with school goals and needs (Exhibit B.10) 

• Mean hours of teacher-reported professional learning 

• Efforts to build structures to support teacher collaboration (Exhibit B.ll) 

• Efforts to build structures to support data use (Exhibit B.12) 

• Presence of district organizational structures (e.g., sub-districts or designated 
staff to support SIG schools) (Exhibit B.14) 

• Presence of SIG partner to support school (Exhibit B.17) 
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Technical Detail 

Stage 2: Construct 
Aggregate Index 

Schools were categorized on each capacity-building indicator (see referenced technical 
approach exhibits for classification schemes). For each indicator, analysts ascribed 
numeric values to the classifications, which were summed to create an aggregate index 
of capacity-building efforts, with a maximum possible score of 10. Most indicators were 
scored according to a binary system, with 0 assigned to the negative category (e.g., "no 
evidence" or "not identified") and 1 to the affirmative category (e.g., "identified"). For 
example, schools rated as having a "purposeful approach to staffing decisions" were 
assigned a 1, and schools rated as having "no evidence of a purposeful approach to 
staffing decisions" were assigned a 0. The only indicators that did not follow this binary 
system were: 

• Mean hours of teacher-reported professional learning. Although scored 
according to a binary system, 0 represents below 111 hours and 1, at or above 

111 hours [Based on the National Longitudinal Study of NCLB, the national 
estimate of the average number of PD hours for teachers in Title 1 schools 
identified for improvement was 111 hours (Taylor et al., 2010)]. 

• Creation of nonteaching staff positions. This indicator was scored using a three- 
point scale with 0 assigned to schools that added non-instructional staff 
positions in Year 1 only, 0.5 assigned to schools that added non-instructional 
staff positions in Year 2 only, and 1 assigned to schools that added non- 
instructional staff in both years. 

• Alignment of professional learning with school goals and needs. This indicator 
was scored using a three-point scale with 0 assigned to schools rated as 
minimally or not aligned, 0.5 assigned to schools rated as moderately aligned, 
and 1 assigned to schools rated as aligned. 

When the scores were complete, the lead site visitor for each school reviewed and 
verified the ratings for his or her school(s). 

Caveats 

This measure reflects a school's overall reported level of effort to build capacity, not 
whether a school actually improved capacity. It does not include information about the 
quality or effectiveness of schools' individual activities to build human capital. In 
addition, this measure may oversimplify and obscure important aspects of the capacity- 
building process in some cases. For example, school leaders might reasonably opt to 
focus on a few leverage points rather than across-the-board efforts. Nevertheless, since 
SIG prescribes numerous activities for its grantees under each model, we believe that 
this aggregate perspective provides information on each school's holistic efforts to build 
human capital that would be lost if we were only to consider each activity in isolation. 

Notes 

Includes 25 core sample schools. 
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Perceptions of Transformational Leadership 


Summary 

This analysis examines the extent to which principals at core subsample schools were perceived 
as being characterized by or engaged in transformational leadership in Year 3 of SIG as part of 
the Year 3 organizational capacity analysis (see Exhibit B.28). 


Technical Detail 

Data Sources 

Year 3 interviews with teachers and instructional coaches, including the following question to 
elicit responses on principal leadership (Note that information may also have been obtained 
through other points in the interview, not just in direct response to the questions listed 
below.): 

Teachers 

• How would you characterize the principal of this school? 

Instructional coaches 

• How would you characterize the principal of this school? 

Year 3 focus groups with teachers and school improvement teams, including the following 
question to elicit responses on principal leadership (Note that information may also have been 
obtained through other points in the focus group, not just in direct response to the questions 
listed below.): 

Teachers 

• How would you characterize the principal of this school? 

School improvement teams 

• (If the principal was replaced at any point during SIG) How does the current principal's 
approach to the school improvement process relate to that of the previous principal? 

Year 3 teacher survey data. 

Stage 1: 
Qualitative 
Data Analysis 
Procedures 

To examine perceptions of the principals' transformational leadership, analysts reviewed all 
coded data associated with school leadership [ IStaff capacity, IPrincipal leadership, 

IGovernance, lOther school leadership, ITeacher leadership], as well as portions of transcripts 
related to the questions identified above. Qualitative data were coded by analysts according to 
the codebook in Appendix A (see Chapter 2 for a description of the coding procedures). Using 
these data, analysts identified 19 principal qualities in Year 1, which were then compared 
against the 21 qualities of effective leadership delineated by Waters et al. (2003) in their meta- 
analysis of principal leadership studies. This process yielded a set of 11 qualities that were 
common to both analyses, of which 8 were qualities characteristic of transformational leaders 
and 3 were qualities uncharacteristic of transformational leaders (Although the labels may 
differ slightly, analysts determined that the content of the qualities emerging from the case 
study data was consistent with that in Waters et al. (2003).): 

Qualities characteristic of transformational leaders 

• Accessible/welcomes input. Easy to approach and communicate with, encourages 
teachers to contribute ideas. Typically described as having an "open door policy." 

• Supportive of staff. Takes into consideration the needs of the teachers and other 
administrators. Typically described as "understanding" and "helpful." 

• Visible/known to school community. Has frequent interactions with staff, students, 
parents, and community. Typically described as a "visible leader." 

• Visionary. Acts as an agent of change and expresses a clear direction for where the 
school is headed. 

• Enthusiastic. Inspires staff and students and takes on a positive attitude. 

• Communicative. Establishes clear communication systems for teachers, staff, and 
parents/community members. 
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Technical Detail 

Stage 1: 
Qualitative 
Data Analysis 
Procedures 

(continued 
from previous 
page) 

• Develops leaders. Gives opportunities to teachers and staff to become leaders and 
increase their capacity as leaders. 

• High expectations. Establishes high expectations for staff and students. Typically 
described as "believing in kids and staff." 

Qualities uncharacteristic of transformational leaders 

• Poor communication. Does not establish clear communication systems for teachers, 
staff, and parents/community members. 

• Unsupportive of staff/bad rapport. Does not take into consideration the needs of 
teachers and staff. 

• Authoritarian. Does not ask for teacher and staff input, and makes decisions 
individually. Typically described as not letting teachers "have a voice." 

Stage 2: 
Teacher 

Survey Data 

Analysis 

Procedures 

Analysts incorporated data from the principal-teacher trust scale, which included six items in 
which teachers were asked about their principal. See Exhibit C.l for a description of this scale. 

Stage 3: 

School 

Classification 

Procedures 

Using the coded interview and focus group data, as well as the teacher survey data, analysts 
categorized principals using the classification scheme on transformational leadership described 
below. When the classifications were complete, the lead site visitor for each school reviewed 
and verified the categorizations for his or her school(s). In cases of disagreement, the analysts 
and lead site visitor returned to the coded data for the particular school(s) in question to 
resolve the disagreement. For this analysis, respondents refer to individuals from all of the 
respondent groups listed above. 

High on continuum 

• Qualitative data: The principal demonstrated at least three of the eight qualities 
related to transformational leadership. (Demonstrated qualities refer to those 
mentioned by at least two respondents); AND 

• Survey data: Principal-teacher trust scale average was at least 0.5 standard deviations 
(0.37) above the Year 1 scale mean (3.09). 

Middle of continuum 

• The principal did not demonstrate evidence to be categorized in the high or low end of 
the continuum. 

Low on continuum 

• Qualitative data: The principal demonstrated none of the eight qualities related to 
transformational leadership; AND 

• Survey data: Principal-teacher trust scale average was at least 0.5 standard deviations 
(0.37) below the Year 1 scale mean (3.09). 

Caveats 

Classifications are based on school staff reports during site visits in Year 3 of SIG and the spring 
2013 teacher survey. The analysts were not able to make direct evaluations of the strength of 
principals' transformational leadership. 

Notes 

Includes 12 core subsample schools. 
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Perceptions of Instructional Leadership 


Summary 

This analysis examines the extent to which principals at core sample schools were perceived as 
being characterized by or engaged in instructional leadership in Year 3 of SIG as part of the 

Year 3 organizational capacity analysis (see Exhibit B.28). 


Technical Detail 

Data Sources 

Year 3 interviews with teachers and instructional coaches, including the following questions to 
elicit responses on principal leadership (Note that information may also have been obtained 
through other points in the interview, not just in direct response to the questions listed 
below.): 

Teachers 

• How would you characterize the principal of this school? 

Instructional coaches 

• How would you characterize the principal of this school? 

Year 3 focus groups with teachers and school improvement teams, including the following 
questions to elicit responses on principal leadership (Note that information may also have 
been obtained through other points in the focus group, not just in direct response to the 
questions listed below.): 

Teachers 

• How would you characterize the principal of this school? 

School improvement teams 

• (If the principal was replaced at any point during SIG) How does the current principal's 
approach to the school improvement process relate to that of the previous principal? 

Year 3 Teacher survey data. 

Stage 1: 
Qualitative 
Data Analysis 
Procedures 

To examine perceptions of principals' instructional leadership, analysts reviewed all coded data 
associated with school leadership [IStaff capacity, ! Principal leadership, IGovernance, lOther 
school leadership, ITeacher leadership], as well as portions of transcripts related to the 
questions identified above. Using these data, analysts identified data on activities associated 
with instructional leadership of the principal (e.g., principals who focused on academics and 
the academic achievement of students, and who were in the classroom providing feedback on 
instruction and/or curriculum), noting how many and which types of respondents provided 
data. Examples of quotations defining the instructional leadership quality include the 
following: 

• "[The principal] knows what kids need at each grade level. If 1 have a question, 1 can 
reliably trust [that the principal] will have a reasonable suggestion. He knows 
instruction at different grade levels." ( instructional coach) 

• "The principal and the assistant principal are a godsend... [The principal] made it so 
that [our school] has a real viable curriculum and we have some people committed to 
rigor, and that's something this school has needed dramatically..." (external support 
provider) 

Stage 2: 
Teacher 

Survey Data 

Analysis 

Procedures 

Analysts incorporated data from the instructional leadership scale, which included seven items 
in which teachers were asked about various attributes of their principal's instructional 
leadership. See Exhibit C.l for a description of this scale. 
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Technical Detail 

Stage 3: 

School 

Classification 

Procedures 

Using the coded interview and focus group data and the teacher survey, analysts categorized 
principals using the classification scheme on instructional leadership described below. When 
the classifications were complete, the lead site visitor for each school reviewed and verified 
the categorizations for his or her school(s). In cases of disagreement, the analysts and lead site 
visitor returned to the coded data for the particular school(s) in question to resolve the 
disagreement. For this analysis, respondents refer to individuals from all of the respondent 
groups listed above. 

High on continuum 

• Qualitative data: At least two school-level respondents described principal activities 
associated with instructional leadership; AND 

• Survey data: Instructional leadership scale average was at least 0.5 standard 
deviations (0.33) above the Year 1 scale mean (3.12). 

Middle of continuum 

• The principal did not demonstrate evidence to be categorized in the high or low end 
of the continuum. 

Low on continuum 

• Qualitative data: Fewer than two school-level respondents mentioned principal 
activities associated with instructional leadership; AND 

• Survey data: Instructional leadership scale average was at least 0.5 standard 
deviations (0.33) below the Year 1 scale mean (3.12). 

Caveats 

Classifications are based on school staff reports during site visits in Year 3 of SIG and the spring 
2013 teacher survey. The analysts were not able to make direct evaluations of principals' 
strength as instructional leaders. 

Notes 

Includes 12 core subsample schools. 
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Summary 

This analysis examines the extent to which principals at core subsample schools 
demonstrated a theory of action in Year 3 of SIG as part of the Year 3 organizational capacity 
analysis (see Exhibit B.28). 


Technical Detail 

Data Sources 

Year 3 interviews with principals, including the following questions, to elicit responses on the 
theory of action (Note that information regarding the theory of action may also have been 
obtained through other points in the interview, not just in direct response to the questions 
listed below. Data on the rationale for selecting particular improvement actions, intended 
outcomes of the improvement actions, and underlying assumptions were generally prompted 
from the questions below.): 

Defining the performance problem 

• What are the key challenges your school is currently facing in your efforts to 
improve student performance? 

• Have the challenges faced by the school in improving student performance changed 
over past the three years? If so, how and why? 

• On previous visits to your school, we heard about performance challenges related to 
[list of challenge areas]. Is this still a challenge? If not, how was that challenge 
overcome? 

• What is your understanding of why this school continues to be low-performing? 

Identifying improvement actions to address the performance problem 

• We want to ask about your approach to making improvements in your school. On 
previous visits, we've talked about a variety of activities your school has 
implemented to address performance challenges. Today, 1 want to hear about the 
key improvement actions or strategies for moving the school forward that you have 
prioritized this year. What are they? Why have you been focusing on these? How do 
the activities relate to the challenges we just discussed? 

• Over the three years of SIG implementation, have the school's key improvement 
actions or strategies changed? If so, how and why? 

• Generally speaking, what do you think your school needs to improve student 
performance (e.g., funding, expertise, staff capacity)? Beyond these steps you're 
already taking, what do you think the school needs to [improve/continue to 
improve] student performance and why? 

Stage 1: 

Defining the 
Concept 

A theory of action, broadly stated, is the implicit or explicit set of operational assumptions 
regarding how the change process will unfold in a given school. To operationalize a principal's 
"theory of action," the study team identified five elements of a theory of action: (1) defining 
the performance problem, (2) identifying a set of improvement actions or primary levers of 
change to address the performance problem, (3) providing a rationale for selecting those 
improvement actions, (4) identifying the intended outcomes of those strategies, and (5) 
explaining the explicit and interrelated assumptions underlying the change process in a 
school. 
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Technical Detail 

Stage 2: 
Qualitative 

Data Analysis 
Procedures 

Using this five-element framework, analysts coded the full principal interview transcripts 
from fall 2012 and spring 2013 to capture whether each of these five elements of a theory of 
action were present in the principal's interviews. This process enabled analysts to examine 
the degree to which principal responses to certain key questions tied together, built on one 
another, and generally aligned with what the principal stated throughout the course of the 
interview, and thereby, allowing the analysts to obtain a comprehensive understanding of if 
and how principals articulated the five components of their theories of action. 

Using the coded principal interviews, analysts summarized the data to produce a narrative 
for each principal's theory of action. When synthesizing the elements of a principal's theory 
of action, they considered all five elements of the theories of action— performance problem, 
actions and primary levers of change, intended outcomes, rationale, and assumptions. This 
narrative included statements such as the following: 

• "The principal described his long-term vision of the school as moving up on the 
district's measure of school performance: 'We hope to become an 'achieving 1 
school.' Overall, the principal expressed a limited view of the challenges at the 
school (low test scores), but was strategic about ways to boost academic 
performance (targeted credit recovery, test preparation, focused professional 
development). However, based on the data it appears that all of the changes made 
throughout Year 3 were primarily driven by the desire to improve. ..performance in 
math and ELA (as well as to improve graduation rates, [high school exit examination] 
pass rates, and other district measures of 'school success'). ..not to enact 
fundamental change in the way the school operates. The principal did mention 
being interested in creating systems and structures that would bring about and 
sustain change without the leadership of a principal, but she didn't delve into the 
details of what these structures are. Note that the extended day was voted to be 
discontinued after SIG (despite the principal's desires to continue the work), and 
professional development would decrease dramatically without SIG funds." 

Stage 3: 

School 

Classification 

Procedures 

For each element of the theory of action, analysts first determined whether the principal 
exhibited the element by examining the extent to which the principal could clearly articulate 
the element. Summary narratives were then used to assess the extent to which the elements 
mutually reinforced each other, or the coherence of the principal's theory of action. Analysts 
then classified the core sample schools based on the classification scheme on theories of 
action described below. When the classifications were complete, the lead site visitor for each 
school reviewed and verified the categorization for his or her school(s). In cases of 
disagreement, the analysts and lead site visitor returned to the coded data for the particular 
school(s) in question to resolve the disagreement. 

High on continuum 

• The principal exhibited at least four of the five elements of a theory of action; AND 

• The principal demonstrated that these elements mutually reinforce each other. 

Mid-High on continuum 

• The principal exhibited at least three of the five elements of a theory of action; AND 

• The principal expressed that some of these elements mutually reinforced one 
another, but not all of the elements reinforced one another or related to the 
perceived performance problem. 
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Technical Detail 

Stage 3: 

School 

Classification 

Procedures 

(continued from 
previous page) 

Mid-Low on continuum 

• The principal exhibited less than three of the five elements of a theory of action; 

AND 

• The principal expressed that most of these elements were externally driven and 
most of the improvement strategies did not mutually reinforce one another or 
address the perceived performance problem. 

Low on continuum 

• The principal exhibited less than three of the five elements of a theory of action; 

AND 

• The principal was not able to or did not articulate certain components of the plan. 

Caveats 

This analysis focuses on principals only and is not a systematic analysis of the theories of 
action held by all respondents. Thus, the analysis is not intended to represent the theory of 
action at the school level. Instead, the analysis is meant to provide insight into the theories of 
action articulated by the principals at 12 core subsample schools. Using principal interviews 
as the primary source for this analysis, however, presents the following limitations: 

• Although principals are typically perceived as the leaders of a school, there may be 
other systems of leadership within a school and respondents outside a school that 
hold their own theories of action (which may or may not align with the theory of 
action set forth by the principal). 

• Principal interviews may not fully represent a principal's theory of action. 

• Theories of action change over time, so it is likely that the theories of action 
articulated by the principals in Year 3 of SIG have since shifted. 

Moreover, because the study team did not ask the principals to explicitly state their theories 
of action, the data are inferred by answers to relevant interview questions. 

Additionally, the Year 3 analysis of theories of action as reported by principals differed from 
the Year 1 analysis in the following ways: 

• In Year 1, we asked slightly different questions about the perceived performance 
problem and identifying improvement actions to address the performance problem. 
We asked questions differently because in Year 3, lead site visitors already had a 
baseline understanding of the school, so there was no need to repeat the same 
questions in Year 3. Instead, we asked questions in a way that would build upon our 
existing understanding at the time of the interview. 

• In Year 1, we only had one data point: spring 2011 principal interviews. In Year 3, we 
had two data points: fall 2012 and spring 2013 principal interviews. Thus, in Year 3, 
we used the interviews from both the fall and the spring. 

Notes 

Includes 12 core subsample schools. 
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Perceptions of Teacher Collaboration 


Summary 

This analysis examines perceptions of teacher collaboration as part of the Year 3 organizational 
capacity analysis (see Exhibit B.28). 


Technical Detail 

Data Sources 

Year 3 interviews with principals, teachers, and instructional coaches, including the following 
questions to elicit responses on teacher collaboration (Note that information may also have 
been obtained through other points in the interview, not just in direct response to the 
questions listed below.): 

Principals 

• To what extent do teachers at this school have the opportunity to collaborate and 
work together? 

Teachers 

• What opportunities do you have to collaborate and work together with other teachers 
at this school? 

Instructional coaches 

• To what extent do teachers at this school have the opportunity to collaborate and 
work together? 

Year 3 focus groups with teachers, including the following questions to elicit responses on 
teacher collaboration (Note that information may also have been obtained through other 
points in the focus group, not just in direct response to the questions listed below.): 

Teachers 

• What opportunities do you have to collaborate and work together with other teachers 
at this school? 

Year 3 teacher survey data. 

Stage 1: 
Qualitative 

Data 

Analysis 

Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see Chapter 

2 for a description of the coding procedures). To examine respondent perceptions of teacher 
collaboration, site visitors responded to the following question in the online data repository 
based on all coded data associated with collaboration [ A Collaboration, SC_Strategiesj: 

• Based on site visit data and the classification instructions below, please classify your 
school's level of teacher collaboration. The primary data sources for your response 
should be interviews with principals, teachers, and instructional coaches, as well as 
focus groups with teachers. 

Each site visitor reviewed the data in the repository for his or her school(s) to ensure the 
accuracy of the entries. Using responses to the above repository question, the analysts 
identified data that reflected the extent to which teachers collaborated at the school, noting 
how many and which types of respondents provided data. Examples of such evidence include: 

• "I'm collaborating a lot more, 1 have someone who's constantly observing my 
instruction, constantly working with me and challenging me and just giving me another 
lens to look at my students. I've been observed several times and gotten feedback and 

1 think I'm much more open to people just being here and seeing what's going on." 
(teacher) 

• "Some departments haven't wanted to collaborate the way others have. At least, 
that's what I've seen. Everything has been kind of 'me' and now we are moving 
towards 'we'— us as a school, what are we going to look at?" ( teacher ) 

• "One of the things we have done this year is a lot of collaboration. We do a lot of 
collaborative lesson planning, a lot of collaborative tutoring and making sure that 
we're up to date on what we're supposed to teach and not just what we're supposed 
to teach but what our outcomes are supposed to be." (teacher) 
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Technical Detail 

Stage 2: 
Teacher 

Survey Data 

Analysis 

Procedures 

Analysts incorporated data from two teacher survey items that measured teacher 
collaboration— Likert-scale items asking about the frequency of two collaborative activities: (1) 
consulting with other teachers about challenges faced in the classroom and (2) sharing lesson 
plans with other teachers. 

Because individual survey items are less precise than survey scales, analysts only differentiated 
schools into two groups: above the survey item mean and below the survey item mean, rather 
than three groups (0.5 standard deviations below the scale mean or lower, within 0.5 standard 
deviations of the scale mean, and 0.5 standard deviations above the scale mean or higher), as 
was done for the survey scales. 

Stage 3: 

School 

Classification 

Procedures 

Using the repository responses to the question above and the teacher survey data, analysts 
categorized schools based on the classification scheme on teacher collaboration described 
below. When the classifications were complete, the lead site visitor for each school was 
required to review and verify the categorizations for his or her school(s). In cases of 
disagreement, the analysts and lead site visitor returned to the coded data for the particular 
school(s) in question to resolve the disagreement. 

Culture of collaboration 

• Qualitative data: At least one of the following respondent groups— at least two 
teachers, the principal, or an instructional coach — reported that collaboration time is 
used for planning lessons, addressing individual student needs, or that time is 
otherwise described as productive; and at least two teachers described the working 
environment as collegial (e.g., "[teachers] are a team," "teachers not only work in 
grade-level teams, but the school staff is like a family— we all work together"); AND 

• Survey data: School means on both teacher collaboration items were above the overall 
Year 1 sample means (3.63 and 3.46). 

Some collaboration 

• Qualitative data : At least one of the following respondent groups— at least two 
teachers, the principal, or an instructional coach — reported that (1) there is formal 
time allotted for collaboration during the school day, but at least one coach or one 
teacher suggested that the time was not used productively or that it was voluntary 
and not well-attended; or (2) there is informal collaboration (e.g., teachers sharing 
lesson plans), but no formal time during the school day that is used for collaboration; 
AND ONE OF THE FOLLOWING: 

• Survey data: School mean on at least one of the two teacher collaboration items was 
below the overall Year 1 sample mean (3.63 and 3.46); OR 

• At least one of the collaboration item means was below the overall Year 1 mean; OR 

• Qualitative data and teacher survey data did not match (e.g., both survey items were 
above or below the overall sample means [3.63 and 3.46], but qualitative data 
indicated "inconsistent collaboration" or a "culture of collaboration"). 
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Technical Detail 

Stage 3: 

School 

Classification 

Procedures 

(continued 
from previous 
page 

Inconsistent collaboration 

• Qualitative data : At least one of the following respondent groups— at least two 
teachers, the principal, or an instructional coach— reported that, while there is formal 
time during the school day allocated for collaboration, it is not used consistently for 
collaboration; OR at least two teachers reported an absence of a culture of 
collaboration, through the following examples: teachers report feeling isolated, 
teachers report that leadership is not supportive of collaboration, or teachers are "off 
in their own rooms doing their own thing"; AND 

• Survey data: School means on both teacher collaboration items were below the overall 
Year 1 sample means (3.63 and 3.46). 

Caveats 

The Year 1 analysis of teacher collaboration included a third survey item which asked teachers 
how often they "discuss what I've learned in professional learning activities with other 
teachers." This item, however, was removed from subsequent administrations of the teacher 
survey to accommodate additional items on professional learning, while minimizing the burden 
on teacher respondents. 

Notes 

Includes 12 core subsample schools. 
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Perceptions of Safety and Orderliness of the School Environment 


Summary 

This analysis examines perceptions of the safety and orderliness of the core subsample 
schools as part of the Year 3 organizational capacity analysis (see Exhibit B.28). 

Technical Detail 

Data Sources 

Year 3 interviews with district administrators, principals, teachers, and instructional coaches 
(No specific questions were asked. Information for this analysis may have been captured at 
multiple points in the interview.) 

Year 3 focus groups with teachers, students, and parents (No specific questions were asked. 
Information for this analysis may have been captured at multiple points in the focus group.) 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine respondent perceptions of 
school safety and orderliness, analysts reviewed all coded data associated with student 
behavior [ A Student behavior] and safety of the school environment [ A School safety]. Using 
these data, analysts identified data on the school environment as reported by respondents, 
noting how many and which types of respondents provided data. 

Stage 2: 

School 

Classification 

Procedures 

Using the coded data, analysts categorized schools based on the classification scheme on 
perceptions of safety and orderliness described below. When the classifications were 
complete, the lead site visitor for each school reviewed and verified the categorizations for 
his or her school(s). In cases of disagreement, the analysts and lead site visitor returned to 
the coded data for the particular school(s) in question to resolve the disagreement. For this 
analysis, respondents refer to individuals from all of the respondent groups listed above. 
Safe/Orderly 

• At least two respondents explicitly described the school as safe or as having none or 
few behavior problems among students, and no respondents disagreed or made 
statements to the contrary. 

Mixed 

• Respondents made contradicting or differing statements about the school's safety 
and about student behavior. 

Unsafe/ Disorderly 

• At least two respondents described a feeling of being unsafe or behavior problems 
among students, and no respondents disagreed or made statements to the contrary. 

Caveats 

These school-level classifications do not include objective, quantitative indicators of student 
behavior or incidents of crime. They are based only on the reported perceptions of interview 
and focus group participants. 

Notes 

Includes 12 core subsample schools. 
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Summary 

This analysis examines perceptions of data use to inform instructional decisions in Year 3 of 

SIG as part of the Year 3 organizational capacity analysis (see Exhibit B.28). 


Technical Detail 

Data Sources 

Year 3 interviews with district administrators, principals, teachers, and instructional coaches, 
including the following questions to elicit responses on the use of data (Note that 
information may also have been obtained through other points in the interview, not just in 
direct response to the questions listed below.): 

District administrators 

• For each school, what instructional improvement strategies were planned or 
implemented this school year? What is the rationale behind these strategies? 

Principals 

• Could you describe the specific improvement strategies your school implemented 
this school year? Flave you prioritized these strategies? 

Teachers 

• Could you describe some of the specific improvement strategies that you know are, 
or will be, adopted at your school this school year as part of SIG? 

Instructional coaches 

• Could you describe some of the specific improvement strategies that you know are, 
or will be, adopted at your school this school year as part of SIG? 

Year 3 focus groups with teachers, including the following question to elicit responses on the 
use of data (Note that information may also have been obtained through other points in the 
focus group, not just in direct response to the questions listed below.): 

Teachers 

• Can you describe some of the specific improvement strategies that were, or will be, 
adopted at your school this school year? 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

To examine the extent to which schools were perceived as using data for instructional 
decisions, analysts reviewed all coded data associated with how data are being used by 
school administrators and staff [$Data use]. Qualitative data were coded by analysts 
according to the codebook in Appendix A (see Chapter 2 for a description of the coding 
procedures). Using these data, analysts identified data on the frequency and purpose of data 
use, as reported by respondents, noting how many and which types of respondents provided 
data. Examples of such evidence include: 

• "Data drives our instruction, all of our instruction. It determines, in most cases in 
most classes, which kids are going to intervention during that 8th period [extended 
day], when they're going, with what frequency, what sort of intensive intervention 
they need." (teacher) 

• "We are constantly using data. One way we use it well this year is around school 
culture, shifting. ..to a system of complete transparency where everything is 
automated. 1 share the report with the school every Friday. ...teachers know they are 
not going to get what you want with a kid if they don't have data about it." 

(principal) 
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Technical Detail 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

(continued from 
previous page) 

• "I've tried to free up morning time to give them an opportunity to be able to input 
data and talk about it, but if no one is leading that instruction who understands how 
to use the data, then it won't be happening properly. We're still finding it a 
challenge to get all the data generated and use it in a way that can have a positive 
impact on practice and student learning." (principal) 

• "This year for reading we're focusing on fluency. ..usually our data suggested we 
needed to address phonics, however [now] our data suggested that we're moving 
beyond that, which is wonderful in grades three through five, finally." (instructional 
coach) 

Stage 2: 

School 

Classification 

Procedures 

Using the coded data, analysts categorized schools based on the classification scheme on 
perceptions of data use described below. When the classifications were complete, the lead 
site visitor for each school reviewed and verified the categorizations for his or her school(s). 

In cases of disagreement, the analysts and lead site visitor returned to the coded data for the 
particular school(s) in question to resolve the disagreement. For this analysis, respondents 
refer to all of the respondent groups listed above. 

High data use 

• At least three respondents described using data frequently and purposefully, such as 
to guide instruction (differentiating instruction) or to identify students to pull out of 
classrooms or for afterschool instruction, or to guide professional learning for 
teachers; AND 

• Respondents reported that the school had specific people who took responsibility 
for data use, such as instructional coaches, data teams, or whole-school processes. 

Medium data use 

• At least three respondents talked about using data to guide instruction or 
professional learning for teachers. 

Low data use 

• Two or fewer respondents discussed reviewing student data; OR 

• Respondents specifically indicated that the school does not use student data. 

Caveats 

This is an aggregate reflection of the perceptions of school respondents on whether data are 
being used to guide instructional practices in the school. 

Notes 

Includes 12 core subsample schools. 
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Perceptions of Locus of Responsibility for Performance Problems 


Summary 

This analysis examines whether respondents perceived the challenges facing their school in 
Year 3 of SIG as within their control (i.e., internal) or not (i.e., external), as part of the Year 3 
organizational capacity analysis (see Exhibit B.28). 


Technical Detail 

Data Sources 

Year 3 interviews with principals and teachers (No specific questions were asked on the locus 
of responsibility for performance problems. Information for this analysis may have been 
captured at multiple points in the interview.) 

Year 3 focus groups with teachers (No specific questions were asked on the locus of 
responsibility for performance problems. Information for this analysis may have been 
captured at multiple points in the focus group.) 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To determine whether respondents 
attributed their schools' performance problems to external versus internal factors, site 
visitors responded to the following question in the online data repository based on the coded 
data (specific codes used are provided in brackets): 

• How did school stakeholders describe the performance problem in their school? 

That is, to what did they attribute their school's history of low performance? Please 
document the different data sources that contribute to your response. This should 
not just be a summary list, but a description of the story that school stakeholders 
tell about persistent failure. Is there a common, shared perspective or different 
interpretations? [“"Problem definition, “"Challenge, C_Behavior, C_Reform history] 

Once a site visitor completed a school's data repository responses, the site visitor who 
accompanied him or her to that school reviewed the responses to ensure their accuracy and 
completeness. Using responses to the question above on the schools' history of low 
performance, analysts identified data on respondent perceptions regarding responsibility for 
their school's performance problems. 

Stage 2: 

School 

Classification 

Procedures 

Using the repository responses to the question above, analysts categorized schools using the 
classification scheme on responsibility for performance problems described below. When the 
classifications were complete, the lead site visitor for each school reviewed and verified the 
categorizations for his or her school(s). In cases of disagreement, the analysts and lead site 
visitor returned to the coded data for the particular school(s) in question to resolve the 
disagreement. For this analysis, respondents refer to at least two teachers or the principal. 

Internal responsibility 

• Respondents described their performance problems as being within the locus of 
control of the adults in the school; OR 

• Respondents described the external context as challenging, but assumed 
responsibility for the school's history of low performance and did not describe these 
external challenges as insurmountable; or respondents described the external 
context in neutral terms, such as "This school is in a neighborhood with high crime." 

Limited internal responsibility 

• Respondents made statements that attributed the performance problems to the 
external context, but also described challenges within the locus of control of the 
adults in the school; OR 

Respondents attributed the school's history of low performance to a mix of internal 
and external factors; or respondents (i.e., at least two teachers or the principal) 
disagreed about the locus of responsibility. 
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Perceptions of Locus of Responsibility for Performance Problems 

(continued from previous page) 



Technical Detail 

Stage 2: 

School 

Classification 

Procedures 

(continued from 
previous page) 

External responsibility 

• Respondents explicitly identified factors external to the school as responsible for the 
performance problem and did not attribute the history of low performance to any 
factors internal to the school (e.g., school culture, instruction, leadership, 
collaboration); or respondents made statements that attributed the performance 
problems to others outside of the school, such as "Parent participation is why we 
are failing. That is the foundation." 

Caveats 

All core subsample schools faced at least some external challenges, whether from the 
immediate context of the school (e.g., isolation or lack of security), lack of district support, 
limited financial resources, or high levels of student poverty and associated challenges. 
Therefore, it would not be surprising for respondents to describe these factors to the site 
visitors. The distinction for this analysis is whether respondents associated performance 
problems with the context. 

In addition, teacher survey data (self-efficacy scale), which were used in the Year 1 analysis of 
locus of responsibility, were not considered in the Year 3 classifications. These items were 
removed from subsequent administrations of the teacher survey to accommodate additional 
items on professional learning, while minimizing the burden on teacher respondents. 

Because the self-efficacy scale data were consistent with the qualitative data in Year 1 across 
all core sample schools, we consider the Year 3 classifications to be comparable to the Year 1 
classifications. 

Notes 

Includes 12 core subsample schools. 
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Organizational Capacity in Core Subsample Schools 


Summary 

This analysis examines the overall level of organizational capacity in core subsample schools 
in Year 3. See Chapter 8 for a discussion of the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 3 interviews with district administrators, principals, teachers, and instructional coaches 
Year 3 focus groups with school improvement teams, teachers, parents, and students 

Year 3 teacher survey data 

Stage 1: 

Identifying 

indicators 

Prior research on school improvement has identified a number of specific variables, or school 
conditions, often associated with schools with higher-than-expected student achievement, 
which together may be indicative of an overall level of organizational capacity. The study 
team identified the following eight indicators to gauge the overall school capacity of each 
core sample school (see Chapter 1 for a discussion of the study's conceptual framework and 
indicators of school capacity): 

• Leadership, specifically the extent to which principals exhibit transformational 
leadership, instructional leadership, and strategic leadership. 

• Coherence, or the degree to which the policies of a school reflect consistent goals; 
the strategies employed are clearly designed to foster achievement of these goals; 
and barriers and detractors from the goals and strategies are systematically 
removed (Honig & Hatch, 2004; Newmann et al., 2001). 

• Clear and shared goals, including a unity of purpose, explicit expectations, and 
shared values for student learning and success (Newmann et al., 2001; Purkey & 

Smith, 1983). 

• Teacher collaboration, often described in the literature as either same-subject 
teachers "identifying a common curriculum, and then analyzing common 
assessment data to make instructional changes" (DuFour, 2004b) or as teachers of 
the same students, but of different subjects, working together (Erb & Doda, 1989; 
Rottier, 2001). 

• Teacher-teacher trust, or the extent to which teachers feel they have mutual respect 
for each other, for those who lead school improvement efforts, and for those who 
are experts at their craft (Consortium on Chicago School Research, 2004). 

• Safe and orderly climate, an environment in which students "have a sense of being 
physically and psychologically safe in their school" (Consortium on Chicago School 
Research— Student-Centered Learning Climate section, 2004). 

• Use of data for instructional decisions, characterized as the monitoring of student 
learning and frequent and transparent use of student outcome data to guide 
instructional decisions (Coburn & Beuschel, 2012; Coburn & Turner, 2012a; Coburn 
& Turner, 2012b). 

• Locus of responsibility, characterized by the way in which school respondents 
attributed the performance problem in their school to factors within their control 
(i.e., internal causes) or outside of their control (i.e., external causes). 
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Technical Detail 

Stage 2: 

Indicator 

Classification 

Procedures 

Schools were first categorized on each leading indicator. Classification schemes related to 
teacher collaboration, safe and orderly climate, use of data for instructional decisions, and 
locus of responsibility are presented in Exhibits B.24, B.25, B.26, and B.27. The procedures for 
the remaining four indicators are as follows: 

Leadership 

Analysts combined the three leadership dimensions— transformational leadership, 
instructional leadership, and strategic leadership (see Exhibits B.21, B.22, and 

B.23)— to create the following categorizations: 

• High level. The principal was classified as "high or mid-high on continuum" across all 
three leadership dimensions (transformational, instructional, and strategic). 

• Moderate level. The principal was neither classified as "high or mid-high on 
continuum" for all three leadership dimensions nor "low or mid-low on continuum" 
for all three leadership dimensions (transformational, instructional, and strategic). 

• Low level. The principal was classified as "low or mid-low on continuum" across all 
three leadership dimensions (transformational, instructional, and strategic). 

Coherence 

Categorizations for coherence were based on three teacher survey items that 
measured programmatic coherence— Likert-scale items asking about the level of 
agreement with the following statements: "Once we start a new program, we follow 
up to make sure that it's working"; "1 worry that we are adopting too many different 
programs and practices in this school" [reverse-coded prior to creating index]; and 
"This school generally chooses only those school improvement activities that fit with 
our improvement goals and strategies." For each item, schools were assigned 
numeric values based on the school mean relative to the overall Year 1 mean (1 = at 
least 0.5 standard deviations below the overall Year 1 mean; 2 = within 0.5 standard 
deviations of the overall Year 1 mean; 3 = at least 0.5 standard deviations above the 
overall Year 1 mean), which were summed to create a coherence index. 

• High level. Received a summative rating of at least 7 out of 9 on the coherence 
index. 

• Moderate level. Received a summative rating of 6 out of 9 on the coherence index. 

• Low level. Received a summative rating of less than 6 out of 9 on the coherence 
index. 

Clear and shared goals 

Analysts used the shared goals scale based teacher survey data (see Appendix C for 
a detailed description of survey scales) to create three classifications, as follows: 

• High level. Shared goals scale average was at least 0.5 standard deviations (0.30) 
above the Year 1 scale mean (3.18). 

• Moderate level. Shared goals scale average was within 0.5 standard deviations (0.30) 
of the Year 1 scale mean (3.18). 

• Low level. Shared goals scale average was at least 0.5 standard deviations (0.30) 
below the Year 1 scale mean (3.18). 
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Technical Detail 

Stage 2: 

Indicator 

Classification 

Procedures 

(continued from 
previous page) 

Teacher-teacher trust 

Analysts used the teacher-teacher trust scale based on the teacher survey data (see 
Appendix C for a detailed description of survey scales) to create three classifications, 
as follows: 

• High level of trust. Teacher trust scale average was at least 0.5 standard deviations 
(0.28) above the Year 1 scale mean (2.93). 

• Average or moderate level of trust. Teacher trust scale average was within 0.5 
standard deviations (0.28) of the Year 1 scale mean (2.93). 

• Low level of trust. Teacher trust scale average was at least 0.5 standard deviations 
(0.28) below the Year 1 scale mean (2.93). 

Stage 3: 

School 

Classification 

Procedures 

For each indicator, analysts assigned numeric values to the classifications (0 for the lowest 
category, 1 for the middle category, and 2 for the highest category), which were summed to 
create an aggregate index of school capacity. Analysts then classified the core sample schools 
based on the Year 1 classification scheme on school capacity described below. In Year 1, 
because there were no natural breaks in the distribution, cutpoints were set to divide the 
schools roughly into thirds. When the classifications were complete, the lead site visitor for 
each school was required to review and verify the categorization for his or her school(s). 

Higher capacity 

• Received a summative rating of at least 10 out of 16 on the school capacity index. 

Moderate capacity 

• Received a summative rating of 8 or 9 out of 16 on the school capacity index. 

Lower capacity 

• Received a summative rating of less than 8 out of 16 on the school capacity index. 

Caveats 

The Year 1 analysis of teacher collaboration included a third survey item which asked 
teachers how often they "discuss what I've learned in professional learning activities with 
other teachers." This item, however, was removed from subsequent administrations of the 
teacher survey to accommodate additional items on professional learning, while minimizing 
the burden on teacher respondents. 

In addition, teacher survey data (self-efficacy scale), which were used in the Year 1 analysis of 
locus of responsibility, were not considered in the Year 3 classifications. These items were 
removed from subsequent administrations of the teacher survey to accommodate additional 
items on professional learning, while minimizing the burden on teacher respondents. 

Because the self-efficacy scale data were consistent with the qualitative data in Year 1 across 
all core sample schools, we consider the Year 3 classifications to be comparable to the Year 1 
classifications. 

Notes 

Includes 12 core subsample schools. 


B-63 


Case Studies of Schools Receiving School Improvement Grants: Final Report 


Exhibit B.29. 

Risk Factors for Sustainability in Core Subsample Schools 


Summary 

This analysis examines perceived challenges to sustaining and continuing school 
improvement in core subsample schools. See Chapter 9 for a discussion of the analysis, 
including analytic results. 


Technical Detail 

Data Sources 

Year 3 interviews with district administrators , principals, teachers, instructional coaches, and 
external support providers, including the following question to elicit responses on threats to 
sustainability (Note that information may also have been obtained through other points in 
the interview, not just in direct response to the questions listed below.): 

District administrators, principals, teachers, instructional coaches, and external support 

providers 

• You mentioned [X] areas in which you've seen improvement at the school. Do you 
expect that the school will be able to sustain and build upon these improvements 
post SIG? Why or why not? 

Year 3 focus groups with school improvement teams and teachers, including the following 
questions to elicit responses on the school's sustainability prospects (Note that information 
may also have been obtained through other points in the focus group, not just in direct 
response to the questions listed below.): 

School improvement teams and teachers 

• Earlier in our conversation, you mentioned seeing improvement in [areas] at the 
school. Do you expect that the school will be able to sustain and build upon these 
improvements post-SIG? Why or why not? 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). Using all coded data associated with 
sustainability [Sustain_Activities, Sustainjmprovements, Sustain_Steps being taken] and 
noting how many and which types of respondents provided data, analysts identified four 
challenges for sustaining and continuing school improvement on which respondents provided 
perceptions: 

• Anticipated turnover in school staff 

• Change in school leadership 

• Lack of district support, particularly with regard to retaining principals and teachers 

• Loss of specific interventions (such as professional learning or extended day 
programs) 

Stage 2: 

Categorization 

Procedures 

Using the coded data, for each challenge, analysts identified whether the explanation was 
perceived as a threat to sustainability using the criteria described below. When the analysis 
was complete, the lead site visitor for each school reviewed and verified the categorizations 
for his or her school(s). In cases of disagreement, the analysts and lead site visitor returned to 
the coded data for the particular school(s) in question to resolve the disagreement. For this 
analysis, respondents refer to individuals from all of the respondent groups listed above. 

Identified as a perceived challenge for sustainability 

• Respondents from at least three respondent groups described this explanation as a 
sustainability challenge. 

Caveats 

This analysis is not a systematic examination of sustainability prospects, but rather an 
aggregate reflection of the perceptions of respondents regarding their school's prospects for 
being able to sustain and build upon improvements. Moreover, this analysis does not look at 
whether any of these occurred or whether improvements were sustained because the study 
did not collect data post-SIG. 

Notes 

Includes 12 core subsample schools. 
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Perceived Prospects for Sustainability in Core Subsample Schools 


Summary 

This analysis examines the extent to which respondents in core subsample schools perceived 
their school's prospects for being able to sustain and build upon improvements. See Chapter 

9 for a discussion of the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 3 interviews with district administrators , principals, teachers, instructional coaches, and 
external support providers, including the following questions to elicit responses on the 
school's prospects to sustain improvements (Note that information may also have been 
obtained through other points in the interview, not just in direct response to the questions 
listed below.): 

District administrators and principals 

• You mentioned [X] areas in which you've seen improvement at the school. Do you 
expect that the school will be able to sustain and build upon these improvements 
post SIG? Why or why not? 

• How will the recent years' progress be maintained and built upon? What steps are 
you taking now to ensure that improvements will be sustained? 

Teachers, instructional coaches, and external support providers 

• You mentioned [X] areas in which you've seen improvement at the school. Do you 
expect that the school will be able to sustain and build upon these improvements 
post SIG? Why or why not? 

Year 3 focus groups with school improvement teams and teachers, including the following 
questions to elicit responses on the school's sustainability prospects (Note that information 
may also have been obtained through other points in the focus group, not just in direct 
response to the questions listed below.): 

School improvement teams 

• Earlier in our conversation, you mentioned seeing improvement in [areas] at the 
school. Do you expect that the school will be able to sustain and build upon these 
improvements post-SIG? Why or why not? 

• How will the recent years' progress be maintained and built upon? What steps or 
actions is your school's [or the district's] leadership taking now to ensure that 
improvements will be sustained? 

• Beyond these steps you're already taking, what do you think the school needs to 
[improve/continue to improve] student performance? Where do you see this school 
in five years? 

Teachers 

• Earlier in our conversation, you mentioned seeing improvement in [areas] at the 
school. Do you expect that the school will be able to sustain and build upon these 
improvements post-SIG? Why or why not? 

Year 3 teacher survey data. 
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(continued from previous page) 



Technical Detail 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). Using all coded data associated with 
sustainability [Sustain_Activities, Sustainjmprovements, Sustain_Steps being taken], 
analysts identified data on respondents' general perceptions regarding sustainability 
prospects, noting how many and which types of respondents provided data. These data 
included statements such as the following: 

• "Well, the sustainability is going to [require that]. ..you do not have a big turnover 
happening. If you can keep the folks that you already spent the last two or three 
years training, implementing the strategies that you need, the culture possibly, it 
would be part of the culture. ..here you're always going to have some turnover 
because of retirement." ( external support provider) 

• "1 don't think that we're at the point yet where like if they remove me and put me 
somewhere else and a few other key people [it'll be sustainable].... We still haven't 
been able to build that bench." ( principal ) 

• "...what we've done with the SIG grant has made such a difference so that the 
biggest fight is to be able to capture or sustain what we can. We're not going to be 
able to keep the whole thing because we don't have the funds to be able to do that, 
but at least we can sustain a piece of it." (district administrator) 

Stage 2: 

Teacher Survey 
Data Analysis 
Procedures 

Analysts incorporated data from the school sustainability index scale, which included six 
items in which teachers were asked the extent to which they agreed or disagreed with 
statements about the school's future. See Exhibit C.l in Appendix C for a description of this 
scale. 

Stage 3: School 
Categorization 
Procedures 

Analysts classified the core subsample schools based on the classification scheme on 
perceived sustainability described below (Perceived risk factors are those which were 
identified as a sustainability challenge by respondents in at least three respondent groups). 
When the classifications were complete, the lead site visitor for each school reviewed and 
verified the categorizations for his or her school(s). In cases of disagreement, the analysts 
and lead site visitor returned to the coded data for the particular school(s) in question to 
resolve the disagreement. For this analysis, respondents refer to individuals from all of the 
respondent groups listed above. 

Perceived as having strong sustainability prospects 

• Qualitative data: No respondent mentioned any of the four risk factors to sustaining 
and continuing school improvement; and no respondent described the school's 
sustainability prospects in negative terms; AND 

• Survey data: Sustainability index scale average was at least 0.5 standard deviations 
(0.26) above the Year 3 scale mean (3.01). 

Perceived as having mixed sustainability prospects 

• Qualitative data: Respondents from at least three respondent groups reported on at 
least one of the four risk factors to sustaining and continuing school improvement, 3 
but respondents' descriptions of sustainability prospects were mostly positive; AND 

• Survey data: Sustainability index scale average was within 0.5 standard deviations 
(0.26) of the scale mean (3.01). 
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Perceived Prospects for Sustainability in Core Subsample Schools 

(continued from previous page) 



Technical Detail 

Stage 3: School 
Categorization 
Procedures 

(continued from 
previous page) 

Perceived as having weak or low sustainability prospects 

• Qualitative data: Respondents from at least three respondent groups reported on at 
least one of the four risk factors to sustaining and continuing school improvement; 3 
and respondents' described the school's sustainability prospects in negative terms; 
AND 

• Survey data: Sustainability index scale average was below the scale mean (3.01) 

0 While theoretically possible, there were no instances in our data where respondents from only one or 
two respondent groups reported on one or more of the four risk factors to sustaining and continuing 
school improvement. 

Caveats 

This analysis is not a systematic examination of sustainability prospects, but rather an 
aggregate reflection of the perceptions of respondents regarding their school's prospects for 
being able to sustain and build upon improvements. Moreover, this analysis does not look at 
whether any of these occurred or whether improvements were sustained because the study 
did not collect data post-SIG. 

Notes 

Includes 12 core subsample schools. 
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Perceptions of Ownership of Reform Efforts 


Summary 

This analysis examines whether respondents in the core subsample schools perceived the 
reform efforts being implemented as within their control. See Chapter 9 for a discussion of 
the analysis, including analytic results. 


Technical Detail 

Data Sources 

Year 2 and 3 interviews with district administrators, principals, teachers, and instructional 
coaches, including the following questions to elicit responses about ownership of reform 
efforts (Note that information may also have been obtained through other points in the 
interview, not just in direct response to the questions listed below.): 

District administrators 

• Year 2: Within the district, how do decisions about individual SIG schools get made? 
Who makes the final decisions about key aspects of the school, such as improvement 
strategies, staffing, curriculum, professional learning, and budget? 

• Year 2: With the school year almost completed, to what extent do you believe the 

SIG schools are making progress? On what do you base this assessment? 

• Year 2: Based on this year's experiences, are you working with the SIG school(s) to 
refine, refocus, or change any improvement strategies next year? If so, why? How? 

• Year 3: If you were to tell the improvement story for each of these schools, what 
would be the story line for each over the past three years? 

• Year 3: Thinking back to where these schools started three years ago, to what extent 
have/has the school(s) made progress since the beginning of SIG? In what areas? 

Why? On what do you base this assessment? 

Principals 

• Year 2: To what extent have the key improvement strategies changed during this 
school year? If so, how? Why or why not? 

• Year 2: With the school year almost completed, to what extent do you believe this 
school is making progress? In what areas? On what do you base this assessment? 

• Year 2: Based on your experiences are you planning to refine or refocus any of the 
improvement strategies next year? If so, why? How? 

• Year 3: If you were to tell the improvement story of this school over the past three 
years, what would the story line be? 

• Year 3: Thinking back to where your school started three years ago, to what extent 
has this school made progress since the beginning of SIG? In what areas? Why? How 
do you know? 

Teachers 

• Year 2: Do you believe the ways in which the school is working to accomplish its goals 
are appropriate given the school's challenges? Why or why not? 

o Do you feel that teachers and students support them? Why or why not? 
o Do you think the school/teachers have the support to implement them well? 

• Year 2: With the school year almost completed, to what extent do you believe this 
school is making progress? In what areas? On what do you base this assessment? 

• Year 3: To what extent has this school made progress since the beginning of SIG? In 
what areas? 

• Year 3: How has your work as a teacher at this school changed over the past three 
years? What caused these changes? 
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Technical Detail 

Data Sources 

(continued from 
previous page) 

Instructional coaches 

• Year 2: To what extent have the improvement strategies changed during this school 
year? How? Why or Why not? 

• Year 2: Do you believe these strategies are appropriate given the school's challenges? 
Why or why not? 

• Year 2: What is your role in implementation of these strategies this school year? 

• Year 2: How is the school monitoring its improvement progress? How do you know if 
your school is improving? 

• Year 3: To what extent has this school made progress since the beginning of SIG? In 
what areas? 

• Year 3: How, if at all, do you expect your work as a coach here to change next year? 
Why? 

Year 2 and 3 focus groups with school improvement teams, teachers and parents, including 
the following questions to elicit responses about ownership of reform efforts (Note that 
information may also have been obtained through other points in the focus group, not just in 
direct response to the questions listed below.): 

School improvement teams 

• Year 2: To what extent have the improvement strategies changed since last year? If 
so, how? Why or why not? 

• Year 2: With the school year almost completed, to what extent do you believe this 
school is making progress? On what do you base this assessment? 

• Year 3: To what extent has this school made progress since the beginning of SIG? In 
what areas? 

Teachers 

• Year 2: Do you believe the ways in which the school is working to accomplish its goals 
are appropriate given the school's challenges? Why or why not? 

• Year 2: How do you learn about different activities taking place or decisions being 
made at the school? How are teachers kept informed? 

• Year 2: Do you think the school is moving in the right direction? In what areas do you 
think there has been positive change? Where do you think there has been a lack of 
change? Why? 

• Year 3: How has your work as teachers at this school changed over the past three 
years? Why? 

• Year 3: To what extent has this school made progress since the beginning of SIG? In 
what areas? 

Parents 

• Year 2: What are your impressions of the improvement activities/changes? How do 
you think they will help improve some of the school's challenges that we've talked 
about? 

• Year 2: What do others in the community think about the school and the education it 
offers? Has that changed in the past year? If so, how? Why? 

• Year 3: Thinking back over the past three years, how has the school changed over 
time? 


B-69 


Case Studies of Schools Receiving School Improvement Grants: Final Report 


Exhibit B.31. 

Perceptions of Ownership of Reform Efforts (continued from previous page) 



Technical Detail 

Stage 1: 
Qualitative 

Data Analysis 
Procedures 

Qualitative data were coded by analysts according to the codebook in Appendix A (see 

Chapter 2 for a description of the coding procedures). To examine whether respondents 
perceived themselves as having ownership of their school's reform efforts, analysts 
responded to the following question in the online data repository based on all Year 2 and 3 
coded data associated with staff buy-in [ A Buy in]: 

• Did respondents provide evidence that teachers had ownership over school reform 
efforts by the third year of the grant? That is, did Year 2 and Year 3 interview 
respondents describe teachers as having the capacity to sustain, spread, or deepen 
reforms, which may be evidenced by teachers contributing ideas, adjusting 
strategies, taking on new responsibilities, or articulating the rationale for changes? At 
a school with evidence of ownership, students or parents may also describe having 
"ownership" over changes. 

Stage 2: 
Classification 

Procedures 

Using the repository responses to the question above, analysts categorized schools using the 
classification scheme on ownership described below. When the classifications were complete, 
the lead site visitor for each school reviewed and verified the categorizations for his or her 
school(s). In cases of disagreement, the analysts and lead site visitor returned to the coded 
data for the particular school(s) in question to resolve the disagreement. 

Evidence of Ownership 

• At least one administrator (district administrator, principal, assistant principal) and at 
least two nonadministrative staff (teacher, instructional coach, parent) described 
teachers, administrators, or other stakeholder groups as contributing ideas, adjusting 
improvement strategies, taking on new responsibilities, and/or articulating the 
rationale for change. 

Caveats 

This analysis is not based on an objective measure of teacher ownership of reform efforts but 
rather an aggregate reflection of respondents' perceptions in Year 3 of SIG of whether 
teachers have the capacity to sustain, spread, and deepen reforms. Moreover, this analysis 
does not look at the actual sustainability of teacher ownership of reform efforts because the 
study did not collect data post-SIG. 

Notes 

Includes 12 core subsample schools. 
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Appendix C. Details of Teacher Survey Analyses 


This appendix provides additional detail on the teacher survey analyses described in Chapter 2, including 
information on teacher scales used in school classifications and various other analyses. 

Teacher Survey Scale Items and Scale Reliability 

For the teacher survey scales, we assessed the quality of the scales by conducting a confirmatory factor 
analysis on the items separately for each scale and by computing the scale reliability (Cronbach's alpha). 
Exhibit C.l shows the reliability and contributing items for each scale. 


Exhibit C.l. 

Teacher Survey Scale Items and Scale Reliability, Spring 2011, 2012, and 2013 

Spring Spring Spring 
2011 2012 2013 

Principal instructional leadership 0.94 0.94 0.95 

Carefully tracks students' academic progress 
Understands how children learn 

Makes clear to the staff his or her expectations for meeting instructional goals 

Sets high expectations for student learning 

Actively monitors the quality of teaching in this school 

Presses teachers to implement what they have learned in professional development 
Knows what is going on in my classroom 

Principal-teacher trust 0.94 0.95 0.96 

The principal has confidence in the expertise of the teachers 
I trust the principal at his or her word 

The principal takes a personal interest in the professional development of teachers 
The principal looks out for the personal welfare of the teachers 
The principal places the needs of children ahead of personal and political interests 
The principal at this school is an effective manager who makes the school run smoothly 

School commitment 0.79 n/a n/a 

I usually look forward to each working day at this school 
I wouldn't want to work in any other school 

I would recommend this school to parents seeking a place for their child 
School resources 0.72 0.68 0.65 

Large class size and/or case load 
Inadequate or substandard facilities 
Too few textbooks and other instructional materials 

Textbooks and instructional materials that are not aligned with state standards 
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Exhibit C.l. 

Teacher Survey Scale Items and Scale Reliability, Spring 2011, 2012, and 2013 

(continued from previous page) 



Spring 

2011 

Spring 

2012 

Spring 

2013 

Shared goals 

0.71 

0.79 

0.75 

At this school, we have a common understanding of the objectives we're trying to achieve with students 

Goals and priorities for this school are clear 

If teachers in this school work hard, we can meet our school's goals for student achievement 

Shared values 

0.86 

0.86 

0.86 

Most teachers at this school have values and philosophies of education that are similar to my own 

Most of my colleagues share a focused vision for student learning 

Student behavior 

0.76 

0.77 

0.79 

Poor student discipline 

Large number of student transfers into this school or your class at various points during the year 

Low student motivation 

Low and/or erratic student attendance 

Teacher-teacher trust 

0.75 

0.83 

0.81 

Teachers in this school are comfortable discussing beliefs about teaching and learning 

Teachers in this school are willing to question one another's views on issues of teaching and learning 

Teachers in this school trust each other 

Sustainability 

n/a 

n/a 

0.88 


I believe that our school will continue to change in positive ways 

School leaders (e.g., principal, department chairs) are committed to continue the efforts to change this school for the 
better 

Teachers in this school are committed to continue the efforts to change this school for the better 
Our school has the systems in place to sustain the changes we've made 

School leaders (e.g., principals, department chairs) will have the resources (e.g., capacity, support) to continue the efforts 
to change this school for the better 

Teachers in this school will have the resources (e.g., capacity, support) to continue the efforts to change this school for the 
better 

Source: SST teacher surveys, spring 2011, 2012, and 2013. 

Notes: Includes 25 core sample schools (13 elementary, 12 high) for the spring 2011 and spring 2012 survey, and 12 core 
subsample schools (5 elementary, 7 high) for the spring 2013 survey. 

Numbers of teachers for each year of data collection (spring 2011, spring 2012, spring 2013): principal instructional leadership 
(A/ = 755, 825, 456); principal-teacher trust (A/ = 744, 799, 443); school commitment (A/ = 765, n/a for 2012 and 2013); school 
resources (A/ = 766, 821, 462); shared goals (A/ = 778, 842, 469); shared values (A/ = 779, 844, 471); student behavior (A/ = 768, 
836, 462); teacher-teacher trust (N = 771, 835, 463); sustainability (N = 339 teachers, n/a for 2011 and 2012). 

The school resources scale for the spring 2012 and spring 2013 survey did not constitute a reliable scale (alpha = 0.68, 0.65) and 
was excluded from all analyses. 
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Classifications Using Survey Data 

Many analyses reported in Chapters 3 to 9 were based on classifying schools with respect to features 
such as their context, reform activities, and practices, several of which relied solely or in part on teacher 
survey data. These classifications based on the survey data use a relative standard rather than a 
criterion-based standard— that is, one comparing schools within the sample to each other versus one 
establishing an absolute threshold that could be used to distinguish schools in any sample. We decided 
on this approach for three reasons. First, using the survey data as a criterion-based measure would 
require us to have an objective threshold to distinguish groups of schools, but no such threshold is 
known for the survey scales used in this study. 48 The schools in the core sample and subsample are not, 
and were not meant to be, a nationally representative sample of schools, so we could not determine 
thresholds from our survey data alone. Second, by design, the survey questions were asked in exactly 
the same way, and in the same order, for all respondents, facilitating valid comparisons among 
respondents. Finally, we determined that there was enough between-school variance (between 10 
percent and 25 percent) on each survey scale to meaningfully distinguish among schools in our sample. 
Exhibit C.2 presents the survey scales and survey index items used in the school classifications, as well as 
the relative cut points, using the 0.5 standard deviations above and below the mean criteria. Note that 
the cut points for all survey scales and survey index items are based on the Year 1 (spring 2011) survey 
data with the exception of the sustainability scale, which was included in only the spring 2013 teacher 
survey and administered to the 12 core subsample schools. Because these analyses are focused in part 
on examining change over time, we used the Year 1 means and standard deviations to determine if 
schools had progressed on these measures. 


Exhibit C.2. 

Teacher Survey Data Used in School Classifications 



N of 


Std. 

Mean - 0.5 

Mean + 0.5 


Respondents 

Mean 

Dev. 

Std. Dev. 

Std. Dev. 

Survey Scales 

Principal Instructional Leadership 

690 

3.12 

0.66 

2.79 

3.45 

Principal-Teacher Trust 

688 

3.09 

0.73 

2.73 

3.46 

Shared Goals 

691 

3.18 

0.60 

2.89 

3.48 

Teacher-Teacher Trust 

692 

2.93 

0.55 

2.65 

3.20 

Sustainability 

339 

3.02 

0.52 

2.76 

3.28 

Survey Index Items 

Programmatic Coherence 






Program follow-up 

683 

2.72 




Adopting too many programs 

687 

2.28 




Programs fit with instructional goals 

682 

2.76 




Teacher Collaboration 






Consult with other teachers about challenges 

679 

3.63 




Share the content of my lesson plans 

681 

3.46 





Source: SST teacher survey, spring 2011; SST teacher survey, spring 2013. 

Notes: Includes 21 of 25 core sample schools; for the sustainability scale, which was only used in spring 2013, 12 core 
subsample schools were included. Four schools were excluded for not meeting the 50 percent response rate threshold on the 


48 Although it is possible to determine a seemingly reasonable threshold for classifications (e.g., 3 corresponded to 
agree in most survey items), categorizations would nonetheless be arbitrary (e.g., why not use 2.5?). 
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spring 2011 teacher survey. 

Survey scales divided schools into three categories: schools whose scale was more than 0.5 standard deviations below the scale 
mean, schools whose scale was within 0.5 standard deviations of the scale mean, and schools whose scale was more than 0.5 
standard deviations above the mean. Survey items used as part of an index for classification were divided into two categories: 
schools at or above the mean, and schools below the mean. 


Classifications Using Survey and Qualitative Data 

Three classifications were developed using survey data in conjunction with respondent interview and 
focus group data: principal leadership, teacher collaboration, and school sustainability prospects. The 
classification process for these analyses involved several steps. First, schools were classified separately 
on the basis of survey data (survey scale or survey index based on individual survey items) and on the 
qualitative data (interviews and focus groups). Then, the survey and qualitative classifications were 
combined to form final classifications. 

When using survey scales to classify schools, we first classified core sample and subsample schools into 
three categories: schools whose scale score was more than 0.5 standard deviations below the overall 
scale mean, schools whose scale score was within 0.5 standard deviations of the overall scale mean, and 
schools whose scale score was more than 0.5 standard deviations above the mean. 49 The following 
examples illustrate in greater detail the ways in which qualitative and survey data were combined to 
create school-level classifications. 

Given the complex nature of analyzing principal leadership— which consists of the three dimensions of 
transformational, instructional, and strategic leadership— we used qualitative data as well as survey 
scale data to create school-level classifications. We first examined interview data for district 
administrators, principals, teachers, and instructional coaches, as well as focus group data for teachers. 
Using these qualitative data, we grouped schools into three preliminary categories for the two 
dimensions of transformational and instructional leadership: 

• Schools where the principal was classified as high on the continuum 

• Schools where the principal was classified as neither high nor low on the continuum (i.e., 
moderate) 

• Schools where the principal was classified as low on the continuum 

We grouped schools into four categories for the third dimension of strategic leadership: 

• Schools where the principal was classified as high on the continuum 

• Schools where the principal was classified as mid-high on the continuum 

• Schools where the principal was classified as mid-low on the continuum 

• Schools where the principal was classified as low on the continuum 


49 Means and standard deviations were taken from the teacher-level data (rather than school-level averages), since 
the survey scales, from which this analysis is derived, measure teacher-level attitudes. Although we are ultimately 
classifying schools, these classifications are based on the average teacher in each school, so that our analysis is 
essentially comparing the average teacher in each school to the average teacher in our full sample of teachers. A 
0.5 standard deviation above and below the mean was set as the threshold as a way to ensure that the 'low' and 
'high' classifications were reasonably different from each other. That is, schools that are in the 'low' group have 
teacher respondents who, on average, responded at least one standard deviation lower than schools in the 'high' 
group. 
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Next, we examined the survey data, grouping schools into three categories (low, medium, and high) 
using the principal instructional leadership and principal-teacher trust scales for the dimensions of 
instructional and transformational leadership based on whether each school's scale score was more 
than 0.5 standard deviations above the overall mean, within 0.5 standard deviations of the mean, or 
more than 0.5 standard deviations below the mean. The third dimension of principal leadership, 
strategic leadership, did not involve survey data. 

Finally, the qualitative data and survey data were combined to determine the three final classifications 
(see Exhibit C.3): 

• Schools where the principal was classified as high (or mid-high) on the continuum across all 
three leadership dimensions (transformational, instructional, and strategic) - final classification 

is high 

• Schools where the principal was classified as low (or mid-low) on the continuum across all three 
leadership dimensions - final classification is low 

• Schools where the principal was classified as neither high (or mid-high) on the continuum 
across all three leadership dimensions nor low (or mid-low) on the continuum across all three 
leadership dimensions - final classification is medium 


Exhibit C.3. 

Leadership Classification Examples Using Qualitative Data and Survey Scales 



Transformational 

Leadership 
Qualitative Rating 

Transformational 

Leadership 
Survey Rating 
(Y1 mean=3.09) 

Instructional 

Leadership 

Qualitative 

Rating 

Instructional 

Leadership 
Survey Rating 
(Y1 mean=3.12) 

Strategic 

Leadership 

Qualitative 

Rating 

Final 

Classification 

Baltimore Bridge 

High 

High 

High 

High 

Mid-high 

High 

Elementary 


(3.63) 


(3.62) 



Paul Bunyan High 

High 

Moderate 

Moderate 

Moderate 

Mid-low 

Medium 



(3.27) 


(3.17) 



McAlliston High 

Moderate 

Moderate 

Moderate 

Moderate 

Mid-high 

Medium 



(3.19) 


(3.01) 



Coral High 

Moderate 

Low 

Moderate 

Moderate 

Mid-low 

Medium 



(2.70) 


(2.96) 



Blizzard Bay 

Moderate 

Low 

Moderate 

Low 

Mid-high 

Medium 

Elementary 


(2.49) 


(2.78) 



Sawbuck 

Low 

Low 

Low 

Low 

Mid-low 

Low 

Elementary 


(2.31) 


(2.20) 




Source: SST respondent interview and focus groups, spring 2013; SST teacher survey, spring 2013. 

Notes: This table is for illustrative purposes only; therefore, it contains only 6 of 12 core subsample schools. All school names 
are pseudonyms. 


Paul Bunyan High's qualitative data, for example, suggested a high level of transformational principal 
leadership. Survey and qualitative data indicated a moderate level of instructional and strategic 
leadership however, resulting in a final principal leadership classification of medium. 
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Because individual survey items are less precise than survey scales, we used survey items differently 
than survey scales. For example, the analysis of teacher collaboration included two survey items, 50 
which asked teachers, "Flow often do you engage in the following activities": 

• Consult about challenges: Consult with other teachers about challenges I am facing in the 
classroom 

• Share lesson plans: Share the content of my lesson plans with other teachers 

These items were coded from never = 1 to often = 4. Although these items measure aspects of teacher 
collaboration, they did not constitute a reliable scale in spring 2012 and spring 2013 (alpha = 0.68, 0.65). 
Thus, we simply classified schools based on whether they were above or below the mean on each. 

We also examined interviews with teachers, coaches, and principals, as well as teacher focus groups (for 
more information, see Exhibit B.24 in Appendix B). Using the qualitative responses in conjunction with 
the teacher survey data, we created the final three classifications: 

• Schools with a culture of collaboration had to have a high or some qualitative rating, and both 
survey items above the mean. 

• Schools with inconsistent collaboration had to have a low or some qualitative rating, and both 
survey items below the mean. 

• Schools with some collaboration had neither a culture or collaboration nor inconsistent 
collaboration. 

For example, Baltimore Bridge Elementary's qualitative data suggested that there is a high level of 
teacher collaboration at the school (see Exhibit C.4). In addition, teachers in this school reported more 
frequent collaborative activities than the average core subsample school, according to the two survey 
items. Therefore, the school was classified as having a culture of collaboration. 


Exhibit C.4. 

Collaboration Classification Examples Using Qualitative Data and Survey Items 



Qualitative 

Data 

Rating 

Survey Data: Consult 
about Challenges 
(Y1 Mean=3.63) 

Survey Data: Share 
Lesson Plans 
(Y1 Mean=3.46) 

Final Classification 

Baltimore Bridge Elementary 

High 

3.86 (above) 

3.63 (above) 

Culture of collaboration 

McAlliston High 

High 

3.80 (above) 

3.60 (above) 

Culture of collaboration 

Sawbuck Elementary 

Some 

3.78 (above) 

3.63 (above) 

Culture of collaboration 

Blizzard Bay Elementary 

Some 

3.86 (above) 

3.41 (below) 

Some collaboration 

Paul Bunyan High 

Some 

3.60 (below) 

3.47 (above) 

Some collaboration 

Coral High 

Some 

3.51 (below) 

3.43 (below) 

Inconsistent collaboration 


Source: SST respondent interview and focus groups, spring 2013; SST teacher survey, spring 2013. 

Notes: This table is for illustrative purposes only; therefore, it contains only 6 of 12 core subsample schools. All school names 
are pseudonyms. 


50 In Year 1, a third item was used to measure teacher collaboration but was not used in subsequent years: 
teachers were asked how often they "discuss what I've learned in professional development activities with other 
teachers." 
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Appendix D. Leading Indicators of School-Level Capacity 


Exhibit D.l. 


Leading Indicators of School-Level Capacity 


Leading 

Indicator 


Principal 

Leadership 


Coherence 


Clear and 
Shared Goals 


Teacher 

Collaboration 


Definition 


Supporting Literature 


Measurement in Case Study 
Schools: 

Data Sources 


Measurement in 
Case Study 
Schools: 
Classifications 


Research and policy stress the central role of the 
principal, especially in leading major change 
efforts (Edmonds, 1979; Rhim et al., 2007; 
Whiteside, 2006). This report examines three 
dimensions of leadership: transformational 
leadership, instructional leadership, and 
strategic leadership. 


Case studies of successful turnaround schools 
consistently point to the role of the principal in 
turnaround efforts (Edmonds, 1979; Herman etal., 2008; 
Purkey & Smith, 1983). One meta-analysis of 70 studies 
of principal leadership found a positive correlation 
between principal leadership (as measured by teacher 
perceptions) and student achievement (Waters, 
Marzano, & McNulty, 2003). 


Programmatic coherence is measured by the 
degree to which the policies of a school reflect 
consistent goals, the strategies employed are 
clearly designed to foster achievement of these 
goals, and barriers and detractors from the goals 
and strategies are systematically removed 
(Honig & Hatch, 2004; Newmann et al., 2001). 


Schools in which goals are clear and shared 
among staff are characterized by a unity of 
purpose, explicit expectations, and shared values 
for student learning and success (Newmann et 
al., 2001; Purkey & Smith, 1983). 


Teacher collaboration is characterized by mutual 
assistance and support within the school context 
(O'Day, Goertz, & Floden, 1995). Often described 
in the literature as either same-subject teachers 
"identifying a common curriculum, developing 
common assessments aligned to that 
curriculum, and then analyzing common 
assessment data to make instructional changes" 
(DuFour, 2004b), or as teachers of the same 
students but of different subjects working 
together (Erb & Doda, 1989; Rottier, 2001). 


Correlational and case studies of schools implementing 
whole-school reforms found that school staff have 
difficulty implementing multiple, unrelated interventions 
(Berends, 2000; Berends, Bodilly, & Kirby, 2002). 
Furthermore, isolated interventions that are not aligned 
with other school or district objectives are less likely to 
achieve desired outcomes than interventions that are 
closely aligned with existing improvement efforts 
(Datnow et al., 2006). 

Studies of schools with higher-than-expected 
achievement found that establishment of a clearly 
defined purpose enables a school to "direct its resources 
and shape its functioning toward the realization of those 
goals" (Purkey & Smith, 1983) and helps to reduce 
student alienation (Newmann, 1981). 

Several studies have found a positive correlation 
between teacher collaboration and student achievement 
(Goddard, Goddard, & Tschannen-Moran, 2007; Herman 
et al., 2008). Teacher collaboration and cooperation 
facilitate improved teacher morale and motivation 
through the sharing of ideas and practices (Corcoran & 
Goertz, 1995). This mutual assistance and support, or the 
"receptivity" of colleagues, plays a role in teachers' daily 
practice (O'Day, Goertz, & Floden, 1995). 


Qualitative data 

• Interviews with principals, 
teachers, and instructional 
coaches 

• Focus groups with teachers 
and school improvement 
teams 

Survey data 

• Principal-teacher trust and 
instructional leadership 
survey scales 

Survey data 

• Three teacher survey items 


Survey data 

• Shared goals survey scale 


Qualitative data 

• Interviews with principals, 
teachers, and instructional 
coaches 

• Focus groups with teachers 

Survey data 

Two teacher survey items 
measuring the frequency 
(never, rarely, sometimes, 
often) of collaborative 
activities 


Higher 

Moderate 

Lower 


Higher 

Moderate 

Lower 


Higher 

Moderate 

Lower 


Culture of 
collaboration 

Some 

collaboration 

Inconsistent 

collaboration 


Number 
of Core 
Sub- 
sample 
Schools, 
Year 1 


Number 
of Core 
Sub- 
sample 
Schools, 
Year 3 
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Exhibit D.l. 

Leading indicators of School-Level Capacity (continued from previous page) 


Leading 

Indicator 

Definition 

Supporting Literature 

Measurement in Case Study 
Schools: 

Data Sources 

Measurement in 
Case Study 

Schools: 

Classifications 

Number 
of Core 
Sub- 
sample 
Schools, 
Year 1 

Number 
of Core 
Sub- 
sample 
Schools, 
Year 3 

Teacher- 

Teachers' sense of trust is referred to as the 

Based on correlational analyses of survey data, Sebring 

Survey data 

Higher 

3 

2 

Teacher Trust 

extent to which teachers feel they have mutual 

and Bryk (2000) found that "in schools that are 

• Teacher-teacher trust 





respect for each other, for those who lead 

improving, where trust and cooperative adult efforts are 

survey scale 

Moderate 

9 

10 


school improvement efforts, and for those who 

strong, students report that they feel safe, sense that 






are experts at their craft (Consortium on Chicago 

teachers care about them, and experience greater 


Lower 

0 

0 


School Research, 2004). 

academic challenge. In contrast, in schools with flat or 







declining test scores, teachers are more likely to state 







that they do not trust one another" (p. 5). 





Safe and 

A safe and orderly climate is an environment in 

A safe school environment characterizes schools that 

Qualitative data 

Safe and orderly 

8 

7 

Orderly Climate 

which students "have a sense of being physically 

have beaten the odds (Bryk et al., 2010; Herman et al., 

• Interviews with district 





and psychologically safe in their school. There 

2008; Johnson & Asera, 1999; U.S. Department of 

administrators, principals, 

Mixed 

2 

5 


are few disruptions due to disciplinary problems, 

Education, 2010a). "Prevailing research suggests that 

teachers, and instructional 





and those that occur are handled firmly and 

students' feelings of safety at school, and problems with 

coaches 

Unsafe and 

2 

0 


fairly" (Consortium on Chicago School 

peer relationships and bullying, are influenced by a broad 


disorderly 




Research— Student-Centered Learning Climate 

array of factors, including students' own attributes, 






section, 2004). 

attributes of their schools, adults with whom students 







interact, families, neighborhoods, and the broader 







society" (Steinberg, Allensworth, & Johnson, 2011). 





Use of Data for 

The use of data for instructional decisions is 

Using data to modify curricular and teaching strategies is 

Qualitative data 

Higher 

5 

8 

Instructional 

characterized as the monitoring of student 

a common feature of turnaround schools (Herman et al., 

• Interviews with district 




Decisions 

learning, and frequent and transparent use of 

2008). Some studies have found that data can help 

administrators, principals, 

Moderate 

4 

4 


student outcome data to guide instructional 

teachers fine-tune their practices and catch learning 

teachers, and instructional 





decisions (Coburn & Beuschel, 2012; Coburn & 

problems before they become intractable, in some cases 

coaches 

Lower 

3 

0 


Turner, 2012a; Coburn & Turner, 2012b). 

diminishing referrals to special education programs 







(Marston et al., 2003; McNamara, 1998; Reschly & 







Starkweather, 1997; Sornson, Frost, & Burns, 2005). 





Locus of 

Locus of responsibility is characterized by the 

Reviews of research have found that schools in which 

Qualitative data 

Internal 

2 

1 

Responsibility 

way in which school respondents attribute the 

teachers exhibit high levels of collective efficacy and take 

• Interviews with principals 





performance problem in their school to factors 

ownership for the challenges facing their schools are 

and teachers 

Limited internal 

8 

10 


within their control (i.e., internal causes) or 

more likely to improve student outcomes (Bandura, 

• Focus groups with teachers 





outside their control (i.e., external causes). 

1993; Goddard, 2001; Goddard, Hoy, & Hoy, 2000; 


External 

2 

1 



Goddard, Hoy, & Hoy, 2004). 
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